diff --git a/papers/atharva_rasane/Algorithm-Encoding.png b/papers/atharva_rasane/Algorithm-Encoding.png new file mode 100644 index 0000000000..8baf14e245 Binary files /dev/null and b/papers/atharva_rasane/Algorithm-Encoding.png differ diff --git a/papers/atharva_rasane/BERT_WATER_MARKING.ipynb b/papers/atharva_rasane/BERT_WATER_MARKING.ipynb new file mode 100644 index 0000000000..909f18b65c --- /dev/null +++ b/papers/atharva_rasane/BERT_WATER_MARKING.ipynb @@ -0,0 +1,100348 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Zbc6iyyjRDhh", + "outputId": "c51037cc-9e81-4255-d3f7-e22259c2f78a" + }, + "outputs": [], + "source": [ + "!pip -q install langchain huggingface_hub transformers sentence_transformers accelerate bitsandbytes" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "id": "OaBDymOCRId8" + }, + "outputs": [], + "source": [ + "import os\n", + "os.environ['HUGGINGFACEHUB_API_TOKEN'] = ''" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "id": "4dhYR_6WRLN2" + }, + "outputs": [], + "source": [ + "# text1 = \"A common use case when generating images is to generate a batch of images, select one image and improve it with a better, more detailed prompt in a second run. To do this, one needs to make each generated image of the batch deterministic. Images are generated by denoising gaussian random noise which can be instantiated by passing a torch generator.\"\n", + "text1 = \"\"\"\n", + "Once upon a time there was a dear little girl who was loved by every one who looked at her, but most of all by her grandmother, and there was nothing that she would not have given to the child. Once she gave her a little cap of red velvet, which suited her so well that she would never wear anything else. So she was always called Little Red Riding Hood.\n", + "\n", + "One day her mother said to her, \"Come, Little Red Riding Hood, here is a piece of cake and a bottle of wine. Take them to your grandmother, she is ill and weak, and they will do her good. Set out before it gets hot, and when you are going, walk nicely and quietly and do not run off the path, or you may fall and break the bottle, and then your grandmother will get nothing. And when you go into her room, don't forget to say, good-morning, and don't peep into every corner before you do it.\"\n", + "\n", + "I will take great care, said Little Red Riding Hood to her mother, and gave her hand on it.\n", + "\n", + "The grandmother lived out in the wood, half a league from the village, and just as Little Red Riding Hood entered the wood, a wolf met her. Little Red Riding Hood did not know what a wicked creature he was, and was not at all afraid of him.\n", + "\n", + "\"Good-day, Little Red Riding Hood,\" said he.\n", + "\n", + "\"Thank you kindly, wolf.\"\n", + "\n", + "\"Whither away so early, Little Red Riding Hood?\"\n", + "\n", + "\"To my grandmother's.\"\n", + "\n", + "\"What have you got in your apron?\"\n", + "\n", + "\"Cake and wine. Yesterday was baking-day, so poor sick grandmother is to have something good, to make her stronger.\"\n", + "\n", + "\"Where does your grandmother live, Little Red Riding Hood?\"\n", + "\n", + "\"A good quarter of a league farther on in the wood. Her house stands under the three large oak-trees, the nut-trees are just below. You surely must know it,\" replied Little Red Riding Hood.\n", + "\n", + "The wolf thought to himself, \"What a tender young creature. What a nice plump mouthful, she will be better to eat than the old woman. I must act craftily, so as to catch both.\" So he walked for a short time by the side of Little Red Riding Hood, and then he said, \"see Little Red Riding Hood, how pretty the flowers are about here. Why do you not look round. I believe, too, that you do not hear how sweetly the little birds are singing. You walk gravely along as if you were going to school, while everything else out here in the wood is merry.\"\n", + "\n", + "Little Red Riding Hood raised her eyes, and when she saw the sunbeams dancing here and there through the trees, and pretty flowers growing everywhere, she thought, suppose I take grandmother a fresh nosegay. That would please her too. It is so early in the day that I shall still get there in good time. And so she ran from the path into the wood to look for flowers. And whenever she had picked one, she fancied that she saw a still prettier one farther on, and ran after it, and so got deeper and deeper into the wood.\n", + "\n", + "Meanwhile the wolf ran straight to the grandmother's house and knocked at the door.\n", + "\n", + "\"Who is there?\"\n", + "\n", + "\"Little Red Riding Hood,\" replied the wolf. \"She is bringing cake and wine. Open the door.\"\n", + "\n", + "\"Lift the latch,\" called out the grandmother, \"I am too weak, and cannot get up.\"\n", + "\n", + "The wolf lifted the latch, the door sprang open, and without saying a word he went straight to the grandmother's bed, and devoured her. Then he put on her clothes, dressed himself in her cap, laid himself in bed and drew the curtains.\n", + "\n", + "Little Red Riding Hood, however, had been running about picking flowers, and when she had gathered so many that she could carry no more, she remembered her grandmother, and set out on the way to her.\n", + "\n", + "She was surprised to find the cottage-door standing open, and when she went into the room, she had such a strange feeling that she said to herself, oh dear, how uneasy I feel to-day, and at other times I like being with grandmother so much.\n", + "\n", + "She called out, \"Good morning,\" but received no answer. So she went to the bed and drew back the curtains. There lay her grandmother with her cap pulled far over her face, and looking very strange.\n", + "\n", + "\"Oh, grandmother,\" she said, \"what big ears you have.\"\n", + "\n", + "\"The better to hear you with, my child,\" was the reply.\n", + "\n", + "\"But, grandmother, what big eyes you have,\" she said.\n", + "\n", + "\"The better to see you with, my dear.\"\n", + "\n", + "\"But, grandmother, what large hands you have.\"\n", + "\n", + "\"The better to hug you with.\"\n", + "\n", + "\"Oh, but, grandmother, what a terrible big mouth you have.\"\n", + "\n", + "\"The better to eat you with.\"\n", + "\n", + "And scarcely had the wolf said this, than with one bound he was out of bed and swallowed up Little Red Riding Hood.\n", + "\n", + "When the wolf had appeased his appetite, he lay down again in the bed, fell asleep and began to snore very loud. The huntsman was just passing the house, and thought to himself, how the old woman is snoring. I must just see if she wants anything.\n", + "\n", + "So he went into the room, and when he came to the bed, he saw that the wolf was lying in it. \"Do I find you here, you old sinner,\" said he. \"I have long sought you.\"\n", + "\n", + "Then just as he was going to fire at him, it occurred to him that the wolf might have devoured the grandmother, and that she might still be saved, so he did not fire, but took a pair of scissors, and began to cut open the stomach of the sleeping wolf.\n", + "\n", + "When he had made two snips, he saw the Little Red Riding Hood shining, and then he made two snips more, and the little girl sprang out, crying, \"Ah, how frightened I have been. How dark it was inside the wolf.\"\n", + "\n", + "And after that the aged grandmother came out alive also, but scarcely able to breathe. Little Red Riding Hood, however, quickly fetched great stones with which they filled the wolf's belly, and when he awoke, he wanted to run away, but the stones were so heavy that he collapsed at once, and fell dead.\n", + "\n", + "Then all three were delighted. The huntsman drew off the wolf's skin and went home with it. The grandmother ate the cake and drank the wine which Little Red Riding Hood had brought, and revived, but Little Red Riding Hood thought to herself, as long as I live, I will never by myself leave the path, to run into the wood, when my mother has forbidden me to do so.It is also related that once when Little Red Riding Hood was again taking cakes to the old grandmother, another wolf spoke to her, and tried to entice her from the path. Little Red Riding Hood, however, was on her guard, and went straight forward on her way, and told her grandmother that she had met the wolf, and that he had said good-morning to her, but with such a wicked look in his eyes, that if they had not been on the public road she was certain he would have eaten her up. \"Well,\" said the grandmother, \"we will shut the door, that he may not come in.\" Soon afterwards the wolf knocked, and cried, \"open the door, grandmother, I am Little Red Riding Hood, and am bringing you some cakes.\" But they did not speak, or open the door, so the grey-beard stole twice or thrice round the house, and at last jumped on the roof, intending to wait until Little Red Riding Hood went home in the evening, and then to steal after her and devour her in the darkness. But the grandmother saw what was in his thoughts. In front of the house was a great stone trough, so she said to the child, take the pail, Little Red Riding Hood. I made some sausages yesterday, so carry the water in which I boiled them to the trough. Little Red Riding Hood carried until the great trough was quite full. Then the smell of the sausages reached the wolf, and he sniffed and peeped down, and at last stretched out his neck so far that he could no longer keep his footing and began to slip, and slipped down from the roof straight into the great trough, and was drowned. But Little Red Riding Hood went joyously home, and no one ever did anything to harm her again.\n", + "\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "id": "4z1BCY7gbGkG" + }, + "outputs": [], + "source": [ + "text1 = \"Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are infeasible for classical computers. Unlike classical computers, which use bits as the fundamental unit of information, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously due to the principles of superposition and entanglement, providing a significant advantage in solving complex computational problems.\"" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 472 + }, + "id": "z_iG32CAERAp", + "outputId": "c652b092-834c-4802-b7bd-d495b94b5744" + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "data": { + "text/plain": [ + "'Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are impossible for classical computers. Unlike quantum computers, which use bits as the fundamental unit of , quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously according to the principles of symmetry and entanglement, providing a significant advantage in solving complex mathematical problems.'" + ] + }, + "execution_count": 30, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from transformers import pipeline, AutoTokenizer, AutoModelForMaskedLM\n", + "import torch\n", + "\n", + "def watermark_text(text, model_name=\"bert-base-uncased\", offset=0):\n", + " # Clean and split the input text\n", + " text = \" \".join(text.split())\n", + " words = text.split()\n", + "\n", + " # Replace every fifth word with [MASK], starting from the offset\n", + " for i in range(offset, len(words)):\n", + " if (i + 1 - offset) % 5 == 0:\n", + " words[i] = '[MASK]'\n", + "\n", + " # Initialize the tokenizer and model, move to GPU if available\n", + " device = 0 if torch.cuda.is_available() else -1\n", + " tokenizer = AutoTokenizer.from_pretrained(model_name)\n", + " model = AutoModelForMaskedLM.from_pretrained(model_name).to(device)\n", + "\n", + " # Initialize the fill-mask pipeline\n", + " classifier = pipeline(\"fill-mask\", model=model, tokenizer=tokenizer, device=device)\n", + "\n", + " # Make a copy of the words list to modify it\n", + " watermarked_words = words.copy()\n", + "\n", + " # Process the text in chunks\n", + " for i in range(offset, len(words), 5):\n", + " chunk = \" \".join(watermarked_words[:i+9])\n", + " if '[MASK]' in chunk:\n", + " try:\n", + " tempd = classifier(chunk)\n", + " except Exception as e:\n", + " print(f\"Error processing chunk '{chunk}': {e}\")\n", + " continue\n", + "\n", + " if tempd:\n", + " templ = tempd[0]\n", + " temps = templ['token_str']\n", + " watermarked_words[i+4] = temps.split()[0]\n", + " # print(\"Done \", i + 1, \"th word\")\n", + "\n", + " # Output the results\n", + " # print(\"Original Text:\")\n", + " # print(text)\n", + " # print(\"Watermark Areas:\")\n", + " # print(\" \".join(words))\n", + " # print(\"Watermarked Text:\")\n", + " # print(\" \".join(watermarked_words))\n", + " return \" \".join(watermarked_words)\n", + "\n", + "# Example usage\n", + "text = \"Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are infeasible for classical computers. Unlike classical computers, which use bits as the fundamental unit of information, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously due to the principles of superposition and entanglement, providing a significant advantage in solving complex computational problems.\"\n", + "watermark_text(text, offset=0)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Im0WJEePMfFD", + "outputId": "921e03a9-d449-47eb-903b-c297327e6011" + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{0: 0.5384615384615384, 1: 0.6153846153846154, 2: 0.5833333333333334, 3: 0.6666666666666666, 4: 0.5833333333333334}\n" + ] + } + ], + "source": [ + "from transformers import pipeline, AutoTokenizer, AutoModelForMaskedLM\n", + "import torch\n", + "\n", + "def watermark_text_and_calculate_matches(text, model_name=\"bert-base-uncased\", max_offset=5):\n", + " # Clean and split the input text\n", + " text = \" \".join(text.split())\n", + " words = text.split()\n", + "\n", + " # Initialize the tokenizer and model, move to GPU if available\n", + " device = 0 if torch.cuda.is_available() else -1\n", + " tokenizer = AutoTokenizer.from_pretrained(model_name)\n", + " model = AutoModelForMaskedLM.from_pretrained(model_name).to(device)\n", + "\n", + " # Initialize the fill-mask pipeline\n", + " classifier = pipeline(\"fill-mask\", model=model, tokenizer=tokenizer, device=device)\n", + "\n", + " # Dictionary to store match ratios for each offset\n", + " match_ratios = {}\n", + "\n", + " # Loop over each offset\n", + " for offset in range(max_offset):\n", + " # Replace every fifth word with [MASK], starting from the offset\n", + " modified_words = words.copy()\n", + " for i in range(offset, len(modified_words)):\n", + " if (i + 1 - offset) % 5 == 0:\n", + " modified_words[i] = '[MASK]'\n", + "\n", + " # Make a copy of the modified words list to work on\n", + " watermarked_words = modified_words.copy()\n", + " total_replacements = 0\n", + " total_matches = 0\n", + "\n", + " # Process the text in chunks\n", + " for i in range(offset, len(modified_words), 5):\n", + " chunk = \" \".join(watermarked_words[:i+9])\n", + " if '[MASK]' in chunk:\n", + " try:\n", + " tempd = classifier(chunk)\n", + " except Exception as e:\n", + " print(f\"Error processing chunk '{chunk}': {e}\")\n", + " continue\n", + "\n", + " if tempd:\n", + " templ = tempd[0]\n", + " temps = templ['token_str']\n", + " original_word = words[i+4]\n", + " replaced_word = temps.split()[0]\n", + " watermarked_words[i+4] = replaced_word\n", + "\n", + " # Increment total replacements and matches\n", + " total_replacements += 1\n", + " if replaced_word == original_word:\n", + " total_matches += 1\n", + "\n", + " # Calculate the match ratio for the current offset\n", + " if total_replacements > 0:\n", + " match_ratio = total_matches / total_replacements\n", + " else:\n", + " match_ratio = 0\n", + "\n", + " match_ratios[offset] = match_ratio\n", + "\n", + " # Return the match ratios for each offset\n", + " return match_ratios\n", + "\n", + "# Example usage\n", + "text = \"Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are infeasible for classical computers. Unlike classical computers, which use bits as the fundamental unit of information, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously due to the principles of superposition and entanglement, providing a significant advantage in solving complex computational problems.\"\n", + "\n", + "# Calculate match ratios\n", + "match_ratios = watermark_text_and_calculate_matches(text, max_offset=5)\n", + "print(match_ratios)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "id": "L8Ht1GNUMgvv" + }, + "outputs": [], + "source": [ + "from scipy.stats import ttest_1samp\n", + "import numpy as np\n", + "\n", + "def check_significant_difference(match_ratios):\n", + " # Extract ratios into a list\n", + " ratios = list(match_ratios.values())\n", + "\n", + " # Find the highest ratio\n", + " highest_ratio = max(ratios)\n", + "\n", + " # Find the average of the other ratios\n", + " other_ratios = [r for r in ratios if r != highest_ratio]\n", + " average_other_ratios = np.mean(other_ratios)\n", + "\n", + " # Perform a t-test to compare the highest ratio to the average of the others\n", + " t_stat, p_value = ttest_1samp(other_ratios, highest_ratio)\n", + "\n", + " # Print the results\n", + " print(f\"Highest Match Ratio: {highest_ratio}\")\n", + " print(f\"Average of Other Ratios: {average_other_ratios}\")\n", + " print(f\"T-Statistic: {t_stat}\")\n", + " print(f\"P-Value: {p_value}\")\n", + "\n", + " # Determine if the difference is statistically significant (e.g., at the 0.05 significance level)\n", + " if p_value < 0.05:\n", + " print(\"The highest ratio is significantly different from the others.\")\n", + " else:\n", + " print(\"The highest ratio is not significantly different from the others.\")\n", + "\n", + " return [highest_ratio, average_other_ratios, t_stat, p_value]\n", + "\n", + "# Example usage\n", + "text = \"Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are infeasible for classical computers. Unlike classical computers, which use bits as the fundamental unit of information, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously due to the principles of superposition and entanglement, providing a significant advantage in solving complex computational problems.\"\n", + "# match_ratios = watermark_text_and_calculate_matches(text, max_offset=5)\n", + "# check_significant_difference(match_ratios)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "uZpox66dMgyW", + "outputId": "293a9391-11f3-4dae-953d-510297d6cc02" + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Done 1 th word\n", + "Done 6 th word\n", + "Done 11 th word\n", + "Done 16 th word\n", + "Done 21 th word\n", + "Done 26 th word\n", + "Done 31 th word\n", + "Done 36 th word\n", + "Done 41 th word\n", + "Done 46 th word\n", + "Done 51 th word\n", + "Done 56 th word\n", + "Done 61 th word\n", + "Original Text:\n", + "Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are infeasible for classical computers. Unlike classical computers, which use bits as the fundamental unit of information, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously due to the principles of superposition and entanglement, providing a significant advantage in solving complex computational problems.\n", + "Watermark Areas:\n", + "Quantum computing is a [MASK] evolving field that leverages [MASK] principles of quantum mechanics [MASK] perform computations that are [MASK] for classical computers. Unlike [MASK] computers, which use bits [MASK] the fundamental unit of [MASK] quantum computers use quantum [MASK] or qubits. Qubits can [MASK] in multiple states simultaneously [MASK] to the principles of [MASK] and entanglement, providing a [MASK] advantage in solving complex [MASK] problems.\n", + "Watermarked Text:\n", + "Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are impossible for classical computers. Unlike quantum computers, which use bits as the fundamental unit of , quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously according to the principles of symmetry and entanglement, providing a significant advantage in solving complex mathematical problems.\n", + "Original Text:\n", + "Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are infeasible for classical computers. Unlike classical computers, which use bits as the fundamental unit of information, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously due to the principles of superposition and entanglement, providing a significant advantage in solving complex computational problems.\n", + "\n", + "Modified Text:\n", + "Quantum computing is example a rapidly evolving field that leverages the principles of quantum mechanics to perform random computations that are impossible for classical computers. Unlike quantum computers, which use bits as the random insert fundamental unit of , quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously according random to the principles of symmetry and entanglement, providing a significant advantage in solving complex mathematical problems.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{0: 0.5714285714285714, 1: 0.5714285714285714, 2: 0.5384615384615384, 3: 0.38461538461538464, 4: 0.7692307692307693}\n", + "Highest Match Ratio: 0.7692307692307693\n", + "Average of Other Ratios: 0.5164835164835164\n", + "T-Statistic: -5.66220858504931\n", + "P-Value: 0.010908789440745323\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "data": { + "text/plain": [ + "[0.7692307692307693,\n", + " 0.5164835164835164,\n", + " -5.66220858504931,\n", + " 0.010908789440745323]" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import random\n", + "\n", + "def randomly_add_words(text, words_to_add, num_words_to_add):\n", + " # Clean and split the input text\n", + " text = \" \".join(text.split())\n", + " words = text.split()\n", + "\n", + " # Insert words randomly into the text\n", + " for _ in range(num_words_to_add):\n", + " # Choose a random position to insert the word\n", + " position = random.randint(0, len(words))\n", + " # Choose a random word to insert\n", + " word_to_insert = random.choice(words_to_add)\n", + " # Insert the word at the random position\n", + " words.insert(position, word_to_insert)\n", + "\n", + " # Join the list back into a string and return the modified text\n", + " modified_text = \" \".join(words)\n", + " return modified_text\n", + "\n", + "# Example usage\n", + "text = \"Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are infeasible for classical computers. Unlike classical computers, which use bits as the fundamental unit of information, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously due to the principles of superposition and entanglement, providing a significant advantage in solving complex computational problems.\"\n", + "words_to_add = [\"example\", \"test\", \"random\", \"insert\"]\n", + "num_words_to_add = 5\n", + "\n", + "# modified_text = randomly_add_words(text, words_to_add, num_words_to_add)\n", + "modified_text = randomly_add_words(watermark_text(text, offset=0), words_to_add, num_words_to_add)\n", + "print(\"Original Text:\")\n", + "print(text)\n", + "print(\"\\nModified Text:\")\n", + "print(modified_text)\n", + "\n", + "match_ratios = watermark_text_and_calculate_matches(modified_text, max_offset=5)\n", + "print(match_ratios)\n", + "check_significant_difference(match_ratios)" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "id": "W22siFz5Mg12" + }, + "outputs": [], + "source": [ + "texts = [\n", + " \"Artificial intelligence (AI) has seen remarkable advancements in recent years, transforming numerous industries. From healthcare to finance, AI technologies are being leveraged to improve efficiency and decision-making. In healthcare, AI algorithms are being used to analyze medical images, predict patient outcomes, and assist in surgery. Finance professionals are using AI for fraud detection, risk management, and algorithmic trading. Despite these advancements, AI also raises ethical concerns, particularly regarding bias and privacy. Ensuring that AI systems are transparent and fair is critical for their continued adoption and trust. As AI continues to evolve, it is essential to consider both its potential benefits and challenges.\",\n", + "\n", + " \"Climate change is one of the most pressing issues facing our planet today. Rising global temperatures, melting ice caps, and increasing frequency of extreme weather events are all indicators of this phenomenon. Scientists warn that without significant action to reduce greenhouse gas emissions, the effects of climate change will become more severe. Renewable energy sources such as solar, wind, and hydro power are being promoted as sustainable alternatives to fossil fuels. Additionally, individuals can make a difference by reducing their carbon footprint through actions like using public transportation, conserving energy, and supporting policies aimed at environmental protection.\",\n", + "\n", + " \"The field of biotechnology is revolutionizing medicine and agriculture. Advances in genetic engineering have enabled scientists to develop crops that are resistant to pests and diseases, as well as produce higher yields. In medicine, biotechnology is being used to create personalized treatments based on an individual's genetic makeup. This approach, known as precision medicine, aims to provide more effective and targeted therapies for various diseases. However, the rapid pace of biotechnological innovation also raises ethical and regulatory questions. It is crucial to balance the benefits of these technologies with the potential risks and ensure that they are used responsibly.\",\n", + "\n", + " \"Quantum computing is poised to revolutionize the world of computing. Unlike classical computers, which use bits to represent data as 0s and 1s, quantum computers use qubits, which can exist in multiple states simultaneously. This allows quantum computers to perform complex calculations much faster than their classical counterparts. Potential applications of quantum computing include cryptography, drug discovery, and optimization problems. However, building a practical and scalable quantum computer remains a significant challenge. Researchers are exploring various approaches, such as superconducting qubits and trapped ions, to overcome these hurdles and bring quantum computing closer to reality.\",\n", + "\n", + " \"The internet of things (IoT) is transforming the way we interact with the world around us. IoT refers to the network of interconnected devices that collect and exchange data. These devices range from smart home appliances to industrial sensors, and their applications are vast. In the home, IoT devices can automate tasks like adjusting the thermostat, turning off lights, and monitoring security systems. In industry, IoT is used to optimize supply chains, monitor equipment health, and improve safety. However, the proliferation of IoT devices also raises concerns about security and privacy. Ensuring that these devices are secure and that data is protected is essential for the continued growth of IoT.\",\n", + "\n", + " \"Renewable energy is gaining momentum as a viable solution to the world's energy needs. Solar, wind, and hydro power are among the most common forms of renewable energy, and they offer a sustainable alternative to fossil fuels. Solar power harnesses energy from the sun using photovoltaic cells, while wind power generates electricity through turbines. Hydropower uses the energy of flowing water to produce electricity. These technologies are being adopted at an increasing rate as countries seek to reduce their carbon emissions and transition to cleaner energy sources. The growth of renewable energy is not without challenges, including the need for improved energy storage solutions and the integration of these technologies into existing power grids.\",\n", + "\n", + " \"The rise of e-commerce has transformed the retail industry. Online shopping has become increasingly popular, offering consumers convenience and a wide range of products at their fingertips. Major e-commerce platforms like Amazon, Alibaba, and eBay have disrupted traditional brick-and-mortar stores, leading to significant changes in consumer behavior. The COVID-19 pandemic further accelerated the shift to online shopping, as lockdowns and social distancing measures limited in-person shopping. While e-commerce offers many benefits, it also presents challenges, such as the need for efficient logistics and concerns about data privacy. As the industry continues to evolve, companies are exploring new technologies like augmented reality and artificial intelligence to enhance the online shopping experience.\",\n", + "\n", + " \"Cybersecurity is a critical concern in today's digital age. With the increasing reliance on technology and the internet, the risk of cyberattacks has grown significantly. Cybercriminals use various methods, such as phishing, ransomware, and malware, to exploit vulnerabilities in systems and steal sensitive information. Organizations must implement robust cybersecurity measures to protect their data and infrastructure. This includes using encryption, multi-factor authentication, and regular security audits. Additionally, individuals can take steps to safeguard their personal information, such as using strong passwords and being cautious of suspicious emails. As cyber threats continue to evolve, staying informed and vigilant is essential for maintaining cybersecurity.\",\n", + "\n", + " \"The field of robotics is advancing rapidly, with applications ranging from manufacturing to healthcare. Industrial robots are used to automate repetitive tasks, improve precision, and increase efficiency in manufacturing processes. In healthcare, robots assist in surgeries, rehabilitation, and patient care. Social robots are being developed to provide companionship and support for the elderly and individuals with disabilities. The integration of artificial intelligence and machine learning has further enhanced the capabilities of robots, enabling them to perform complex tasks and adapt to new situations. However, the rise of robotics also raises ethical and societal questions, such as the impact on employment and the need for responsible development and use of these technologies.\",\n", + "\n", + " \"Space exploration has captured the imagination of humanity for centuries. Recent advancements in technology have made space missions more feasible and ambitious. Private companies like SpaceX and Blue Origin are playing a significant role in this new era of space exploration. SpaceX's successful launches and plans for Mars colonization have reignited interest in space travel. NASA and other space agencies are also focusing on missions to the Moon, Mars, and beyond. The development of new propulsion systems, space habitats, and life support technologies are critical for the success of these missions. While space exploration holds great promise, it also presents challenges, including the need for international cooperation, funding, and addressing the environmental impact of space activities.\",\n", + "\n", + " \"Climate change is driving the need for sustainable agriculture practices. Traditional farming methods often rely on chemical fertilizers and pesticides, which can harm the environment and human health. Sustainable agriculture aims to reduce the negative impact of farming by promoting practices that conserve resources, protect biodiversity, and improve soil health. Techniques such as crop rotation, cover cropping, and organic farming are being adopted by farmers worldwide. Additionally, advances in agricultural technology, such as precision farming and vertical farming, are helping to increase efficiency and reduce waste. By embracing sustainable agriculture, we can ensure food security for future generations while protecting the planet.\",\n", + "\n", + " \"The rise of electric vehicles (EVs) is transforming the automotive industry. EVs offer a cleaner and more sustainable alternative to traditional gasoline-powered vehicles, with lower emissions and reduced dependence on fossil fuels. Major automakers are investing heavily in EV technology, and the market for electric cars is growing rapidly. Advances in battery technology are improving the range and performance of EVs, making them more practical for everyday use. Governments around the world are also supporting the transition to electric vehicles through incentives, subsidies, and the development of charging infrastructure. While challenges remain, such as the need for widespread charging stations and the environmental impact of battery production, the future of transportation is increasingly electric.\",\n", + "\n", + " \"Artificial intelligence (AI) is transforming the field of education. AI-powered tools and platforms are being used to personalize learning, automate administrative tasks, and provide real-time feedback to students. Personalized learning systems use AI algorithms to analyze student performance and tailor instruction to individual needs. This approach can help improve student outcomes by addressing learning gaps and providing targeted support. AI is also being used to create adaptive assessments, intelligent tutoring systems, and virtual learning environments. While AI in education offers many benefits, it also raises questions about data privacy, the role of teachers, and the need for equitable access to technology. As AI continues to evolve, it has the potential to revolutionize the way we teach and learn.\",\n", + "\n", + " \"The field of renewable energy is experiencing significant growth as countries seek to reduce their carbon emissions and transition to cleaner energy sources. Solar, wind, and hydro power are among the most common forms of renewable energy, and they offer a sustainable alternative to fossil fuels. Solar power harnesses energy from the sun using photovoltaic cells, while wind power generates electricity through turbines. Hydropower uses the energy of flowing water to produce electricity. These technologies are being adopted at an increasing rate, driven by advancements in technology, falling costs, and supportive government policies. The growth of renewable energy is not without challenges, including the need for improved energy storage solutions and the integration of these technologies into existing power grids.\",\n", + "\n", + " \"The COVID-19 pandemic has had a profound impact on the world, affecting nearly every aspect of daily life. The pandemic has led to widespread illness, loss of life, and economic disruption. Healthcare systems have been stretched to their limits, and the need for effective treatments and vaccines has become paramount. Scientists and researchers have worked tirelessly to develop vaccines and treatments for COVID-19, leading to the rapid development and distribution of several effective vaccines. The pandemic has also highlighted the importance of public health measures, such as social distancing, mask-wearing, and hand hygiene. As the world continues to grapple with the pandemic, efforts to prevent future outbreaks and improve global health infrastructure are essential.\",\n", + "\n", + " \"The concept of smart cities is gaining traction as urban areas look for ways to improve efficiency, sustainability, and quality of life for residents. Smart cities leverage technology and data to optimize city services, such as transportation, energy, and waste management. For example, smart traffic management systems can reduce congestion and improve air quality by adjusting traffic signals in real-time based on traffic flow. Smart grids can enhance energy efficiency by balancing supply and demand and integrating renewable energy sources. Additionally, smart waste management systems use sensors to monitor waste levels and optimize collection routes. While smart cities offer many benefits, they also raise concerns about data privacy, cybersecurity, and the need for equitable access to technology.\",\n", + "\n", + " \"The field of biotechnology is revolutionizing medicine and agriculture. Advances in genetic engineering have enabled scientists to develop crops that are resistant to pests and diseases, as well as produce higher yields. In medicine, biotechnology is being used to create personalized treatments based on an individual's genetic makeup. This approach, known as precision medicine, aims to provide more effective and targeted therapies for various diseases. However, the rapid pace of biotechnological innovation also raises ethical and regulatory questions. It is crucial to balance the benefits of these technologies with the potential risks and ensure that they are used responsibly.\",\n", + "\n", + " \"The rise of renewable energy is transforming the global energy landscape. Solar, wind, and hydro power are among the most common forms of renewable energy, and they offer a sustainable alternative to fossil fuels. Solar power harnesses energy from the sun using photovoltaic cells, while wind power generates electricity through turbines. Hydropower uses the energy of flowing water to produce electricity. These technologies are being adopted at an increasing rate as countries seek to reduce their carbon emissions and transition to cleaner energy sources. The growth of renewable energy is not without challenges, including the need for improved energy storage solutions and the integration of these technologies into existing power grids.\",\n", + "\n", + " \"The field of cybersecurity is becoming increasingly important as our reliance on technology and the internet grows. Cyberattacks can have devastating consequences, including the theft of sensitive information, financial loss, and damage to an organization's reputation. Cybercriminals use various methods, such as phishing, ransomware, and malware, to exploit vulnerabilities in systems. Organizations must implement robust cybersecurity measures to protect their data and infrastructure. This includes using encryption, multi-factor authentication, and regular security audits. Additionally, individuals can take steps to safeguard their personal information, such as using strong passwords and being cautious of suspicious emails. As cyber threats continue to evolve, staying informed and vigilant is essential for maintaining cybersecurity.\",\n", + "\n", + " \"The rise of e-commerce has transformed the retail industry. Online shopping has become increasingly popular, offering consumers convenience and a wide range of products at their fingertips. Major e-commerce platforms like Amazon, Alibaba, and eBay have disrupted traditional brick-and-mortar stores, leading to significant changes in consumer behavior. The COVID-19 pandemic further accelerated the shift to online shopping, as lockdowns and social distancing measures limited in-person shopping. While e-commerce offers many benefits, it also presents challenges, such as the need for efficient logistics and concerns about data privacy. As the industry continues to evolve, companies are exploring new technologies like augmented reality and artificial intelligence to enhance the online shopping experience.\",\n", + "\n", + " \"Artificial intelligence (AI) is transforming the field of healthcare. AI-powered tools and platforms are being used to analyze medical images, predict patient outcomes, and assist in surgery. In radiology, AI algorithms can help detect abnormalities in medical images, such as tumors or fractures, with high accuracy. In predictive analytics, AI can analyze patient data to identify individuals at risk of developing certain conditions, allowing for early intervention and personalized treatment plans. AI is also being used in robotic surgery, where it can enhance precision and reduce the risk of complications. While AI in healthcare offers many benefits, it also raises questions about data privacy, the role of healthcare professionals, and the need for regulatory oversight.\",\n", + "\n", + " \"The field of renewable energy is experiencing significant growth as countries seek to reduce their carbon emissions and transition to cleaner energy sources. Solar, wind, and hydro power are among the most common forms of renewable energy, and they offer a sustainable alternative to fossil fuels. Solar power harnesses energy from the sun using photovoltaic cells, while wind power generates electricity through turbines. Hydropower uses the energy of flowing water to produce electricity. These technologies are being adopted at an increasing rate, driven by advancements in technology, falling costs, and supportive government policies. The growth of renewable energy is not without challenges, including the need for improved energy storage solutions and the integration of these technologies into existing power grids.\",\n", + "\n", + " \"The COVID-19 pandemic has had a profound impact on the world, affecting nearly every aspect of daily life. The pandemic has led to widespread illness, loss of life, and economic disruption. Healthcare systems have been stretched to their limits, and the need for effective treatments and vaccines has become paramount. Scientists and researchers have worked tirelessly to develop vaccines and treatments for COVID-19, leading to the rapid development and distribution of several effective vaccines. The pandemic has also highlighted the importance of public health measures, such as social distancing, mask-wearing, and hand hygiene. As the world continues to grapple with the pandemic, efforts to prevent future outbreaks and improve global health infrastructure are essential.\",\n", + "\n", + " \"The concept of smart cities is gaining traction as urban areas look for ways to improve efficiency, sustainability, and quality of life for residents. Smart cities leverage technology and data to optimize city services, such as transportation, energy, and waste management. For example, smart traffic management systems can reduce congestion and improve air quality by adjusting traffic signals in real-time based on traffic flow. Smart grids can enhance energy efficiency by balancing supply and demand and integrating renewable energy sources. Additionally, smart waste management systems use sensors to monitor waste levels and optimize collection routes. While smart cities offer many benefits, they also raise concerns about data privacy, cybersecurity, and the need for equitable access to technology.\",\n", + "\n", + " \"The field of biotechnology is revolutionizing medicine and agriculture. Advances in genetic engineering have enabled scientists to develop crops that are resistant to pests and diseases, as well as produce higher yields. In medicine, biotechnology is being used to create personalized treatments based on an individual's genetic makeup. This approach, known as precision medicine, aims to provide more effective and targeted therapies for various diseases. However, the rapid pace of biotechnological innovation also raises ethical and regulatory questions. It is crucial to balance the benefits of these technologies with the potential risks and ensure that they are used responsibly.\",\n", + "\n", + " \"The rise of renewable energy is transforming the global energy landscape. Solar, wind, and hydro power are among the most common forms of renewable energy, and they offer a sustainable alternative to fossil fuels. Solar power harnesses energy from the sun using photovoltaic cells, while wind power generates electricity through turbines. Hydropower uses the energy of flowing water to produce electricity. These technologies are being adopted at an increasing rate as countries seek to reduce their carbon emissions and transition to cleaner energy sources. The growth of renewable energy is not without challenges, including the need for improved energy storage solutions and the integration of these technologies into existing power grids.\",\n", + "\n", + " \"The field of cybersecurity is becoming increasingly important as our reliance on technology and the internet grows. Cyberattacks can have devastating consequences, including the theft of sensitive information, financial loss, and damage to an organization's reputation. Cybercriminals use various methods, such as phishing, ransomware, and malware, to exploit vulnerabilities in systems. Organizations must implement robust cybersecurity measures to protect their data and infrastructure. This includes using encryption, multi-factor authentication, and regular security audits. Additionally, individuals can take steps to safeguard their personal information, such as using strong passwords and being cautious of suspicious emails. As cyber threats continue to evolve, staying informed and vigilant is essential for maintaining cybersecurity.\",\n", + "\n", + " \"The rise of e-commerce has transformed the retail industry. Online shopping has become increasingly popular, offering consumers convenience and a wide range of products at their fingertips. Major e-commerce platforms like Amazon, Alibaba, and eBay have disrupted traditional brick-and-mortar stores, leading to significant changes in consumer behavior. The COVID-19 pandemic further accelerated the shift to online shopping, as lockdowns and social distancing measures limited in-person shopping. While e-commerce offers many benefits, it also presents challenges, such as the need for efficient logistics and concerns about data privacy. As the industry continues to evolve, companies are exploring new technologies like augmented reality and artificial intelligence to enhance the online shopping experience.\",\n", + "\n", + " \"Artificial intelligence (AI) is transforming the field of healthcare. AI-powered tools and platforms are being used to analyze medical images, predict patient outcomes, and assist in surgery. In radiology, AI algorithms can help detect abnormalities in medical images, such as tumors or fractures, with high accuracy. In predictive analytics, AI can analyze patient data to identify individuals at risk of developing certain conditions, allowing for early intervention and personalized treatment plans. AI is also being used in robotic surgery, where it can enhance precision and reduce the risk of complications. While AI in healthcare offers many benefits, it also raises questions about data privacy, the role of healthcare professionals, and the need for regulatory oversight.\",\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "aupVsU8ObJM1", + "outputId": "4f4120f4-34b1-47f6-88c5-8d0dc957edae" + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Done 1 th word\n", + "Done 6 th word\n", + "Done 11 th word\n", + "Done 16 th word\n", + "Done 21 th word\n", + "Done 26 th word\n", + "Done 31 th word\n", + "Done 36 th word\n", + "Done 41 th word\n", + "Done 46 th word\n", + "Done 51 th word\n", + "Done 56 th word\n", + "Done 61 th word\n", + "Done 66 th word\n", + "Done 71 th word\n", + "Done 76 th word\n", + "Done 81 th word\n", + "Done 86 th word\n", + "Done 91 th word\n", + "Done 96 th word\n", + "Original Text:\n", + "Artificial intelligence (AI) has seen remarkable advancements in recent years, transforming numerous industries. From healthcare to finance, AI technologies are being leveraged to improve efficiency and decision-making. In healthcare, AI algorithms are being used to analyze medical images, predict patient outcomes, and assist in surgery. Finance professionals are using AI for fraud detection, risk management, and algorithmic trading. Despite these advancements, AI also raises ethical concerns, particularly regarding bias and privacy. Ensuring that AI systems are transparent and fair is critical for their continued adoption and trust. As AI continues to evolve, it is essential to consider both its potential benefits and challenges.\n", + "Watermark Areas:\n", + "Artificial intelligence (AI) has [MASK] remarkable advancements in recent [MASK] transforming numerous industries. From [MASK] to finance, AI technologies [MASK] being leveraged to improve [MASK] and decision-making. In healthcare, [MASK] algorithms are being used [MASK] analyze medical images, predict [MASK] outcomes, and assist in [MASK] Finance professionals are using [MASK] for fraud detection, risk [MASK] and algorithmic trading. Despite [MASK] advancements, AI also raises [MASK] concerns, particularly regarding bias [MASK] privacy. Ensuring that AI [MASK] are transparent and fair [MASK] critical for their continued [MASK] and trust. As AI [MASK] to evolve, it is [MASK] to consider both its [MASK] benefits and challenges.\n", + "Watermarked Text:\n", + "Artificial intelligence (AI) has made remarkable advancements in recent years transforming numerous industries. From manufacturing to finance, AI technologies are being leveraged to improve performance and decision-making. In healthcare, ai algorithms are being used to analyze medical images, predict patient outcomes, and assist in how Finance professionals are using them for fraud detection, risk management and algorithmic trading. Despite these advancements, AI also raises ethical concerns, particularly regarding bias and privacy. Ensuring that AI algorithms are transparent and fair is critical for their continued integrity and trust. As AI continues to evolve, it is important to consider both its potential benefits and challenges.\n", + "Original Text:\n", + "Artificial intelligence (AI) has seen remarkable advancements in recent years, transforming numerous industries. From healthcare to finance, AI technologies are being leveraged to improve efficiency and decision-making. In healthcare, AI algorithms are being used to analyze medical images, predict patient outcomes, and assist in surgery. Finance professionals are using AI for fraud detection, risk management, and algorithmic trading. Despite these advancements, AI also raises ethical concerns, particularly regarding bias and privacy. Ensuring that AI systems are transparent and fair is critical for their continued adoption and trust. As AI continues to evolve, it is essential to consider both its potential benefits and challenges.\n", + "\n", + "Modified Text:\n", + "Artificial intelligence (AI) has made remarkable advancements in recent years transforming numerous industries. From manufacturing to finance, AI technologies are being leveraged to improve performance and decision-making. In healthcare, ai algorithms are being used to analyze medical images, predict patient outcomes, random and assist in how Finance professionals are using them for fraud example detection, risk management and algorithmic trading. Despite these advancements, AI also raises ethical concerns, particularly regarding bias and privacy. Ensuring that example AI algorithms are transparent and fair test is critical for their continued integrity and trust. As AI continues to evolve, it is random important to consider both its potential benefits and challenges.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{0: 0.6190476190476191, 1: 0.3333333333333333, 2: 0.42857142857142855, 3: 0.2857142857142857, 4: 0.55}\n", + "Highest Match Ratio: 0.6190476190476191\n", + "Average of Other Ratios: 0.3994047619047619\n", + "T-Statistic: -3.765894344306259\n", + "P-Value: 0.032757613277666235\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "data": { + "text/plain": [ + "[0.6190476190476191,\n", + " 0.3994047619047619,\n", + " -3.765894344306259,\n", + " 0.032757613277666235]" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "text = texts[0]\n", + "words_to_add = [\"example\", \"test\", \"random\", \"insert\"]\n", + "num_words_to_add = 5\n", + "\n", + "# modified_text = randomly_add_words(text, words_to_add, num_words_to_add)\n", + "modified_text = randomly_add_words(watermark_text(text, offset=0), words_to_add, num_words_to_add)\n", + "print(\"Original Text:\")\n", + "print(text)\n", + "print(\"\\nModified Text:\")\n", + "print(modified_text)\n", + "\n", + "match_ratios = watermark_text_and_calculate_matches(modified_text, max_offset=5)\n", + "print(match_ratios)\n", + "check_significant_difference(match_ratios)" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "YDaqULs0MeUI", + "outputId": "a278c87d-20c0-44f2-db3d-213dcd3b2bc0" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "___________________________________________________________________________________________________________________________\n", + "Doing 1\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Done 1 th word\n", + "Done 6 th word\n", + "Done 11 th word\n", + "Done 16 th word\n", + "Done 21 th word\n", + "Done 26 th word\n", + "Done 31 th word\n", + "Done 36 th word\n", + "Done 41 th word\n", + "Done 46 th word\n", + "Done 51 th word\n", + "Done 56 th word\n", + "Done 61 th word\n", + "Done 66 th word\n", + "Done 71 th word\n", + "Done 76 th word\n", + "Done 81 th word\n", + "Done 86 th word\n", + "Done 91 th word\n", + "Done 96 th word\n", + "Original Text:\n", + "Artificial intelligence (AI) has seen remarkable advancements in recent years, transforming numerous industries. From healthcare to finance, AI technologies are being leveraged to improve efficiency and decision-making. In healthcare, AI algorithms are being used to analyze medical images, predict patient outcomes, and assist in surgery. Finance professionals are using AI for fraud detection, risk management, and algorithmic trading. Despite these advancements, AI also raises ethical concerns, particularly regarding bias and privacy. Ensuring that AI systems are transparent and fair is critical for their continued adoption and trust. As AI continues to evolve, it is essential to consider both its potential benefits and challenges.\n", + "Watermark Areas:\n", + "Artificial intelligence (AI) has [MASK] remarkable advancements in recent [MASK] transforming numerous industries. From [MASK] to finance, AI technologies [MASK] being leveraged to improve [MASK] and decision-making. In healthcare, [MASK] algorithms are being used [MASK] analyze medical images, predict [MASK] outcomes, and assist in [MASK] Finance professionals are using [MASK] for fraud detection, risk [MASK] and algorithmic trading. Despite [MASK] advancements, AI also raises [MASK] concerns, particularly regarding bias [MASK] privacy. Ensuring that AI [MASK] are transparent and fair [MASK] critical for their continued [MASK] and trust. As AI [MASK] to evolve, it is [MASK] to consider both its [MASK] benefits and challenges.\n", + "Watermarked Text:\n", + "Artificial intelligence (AI) has made remarkable advancements in recent years transforming numerous industries. From manufacturing to finance, AI technologies are being leveraged to improve performance and decision-making. In healthcare, ai algorithms are being used to analyze medical images, predict patient outcomes, and assist in how Finance professionals are using them for fraud detection, risk management and algorithmic trading. Despite these advancements, AI also raises ethical concerns, particularly regarding bias and privacy. Ensuring that AI algorithms are transparent and fair is critical for their continued integrity and trust. As AI continues to evolve, it is important to consider both its potential benefits and challenges.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5238095238095238\n", + "Average of Other Ratios: 0.4339285714285714\n", + "T-Statistic: -16.777777777777764\n", + "P-Value: 0.0004610425326555902\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5789473684210527\n", + "Average of Other Ratios: 0.4375\n", + "T-Statistic: -11.315789473684214\n", + "P-Value: 0.0014802633181549077\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 2\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Done 1 th word\n", + "Done 6 th word\n", + "Done 11 th word\n", + "Done 16 th word\n", + "Done 21 th word\n", + "Done 26 th word\n", + "Done 31 th word\n", + "Done 36 th word\n", + "Done 41 th word\n", + "Done 46 th word\n", + "Done 51 th word\n", + "Done 56 th word\n", + "Done 61 th word\n", + "Done 66 th word\n", + "Done 71 th word\n", + "Done 76 th word\n", + "Done 81 th word\n", + "Done 86 th word\n", + "Done 91 th word\n", + "Original Text:\n", + "Climate change is one of the most pressing issues facing our planet today. Rising global temperatures, melting ice caps, and increasing frequency of extreme weather events are all indicators of this phenomenon. Scientists warn that without significant action to reduce greenhouse gas emissions, the effects of climate change will become more severe. Renewable energy sources such as solar, wind, and hydro power are being promoted as sustainable alternatives to fossil fuels. Additionally, individuals can make a difference by reducing their carbon footprint through actions like using public transportation, conserving energy, and supporting policies aimed at environmental protection.\n", + "Watermark Areas:\n", + "Climate change is one [MASK] the most pressing issues [MASK] our planet today. Rising [MASK] temperatures, melting ice caps, [MASK] increasing frequency of extreme [MASK] events are all indicators [MASK] this phenomenon. Scientists warn [MASK] without significant action to [MASK] greenhouse gas emissions, the [MASK] of climate change will [MASK] more severe. Renewable energy [MASK] such as solar, wind, [MASK] hydro power are being [MASK] as sustainable alternatives to [MASK] fuels. Additionally, individuals can [MASK] a difference by reducing [MASK] carbon footprint through actions [MASK] using public transportation, conserving [MASK] and supporting policies aimed [MASK] environmental protection.\n", + "Watermarked Text:\n", + "Climate change is one of the most pressing issues facing our planet today. Rising global temperatures, melting ice caps, and increasing frequency of extreme weather events are all indicators of this phenomenon. Scientists warn people without significant action to reduce greenhouse gas emissions, the consequences of climate change will be more severe. Renewable energy sources such as solar, wind, and hydro power are being considered as sustainable alternatives to fossil fuels. Additionally, individuals can make a difference by reducing their carbon footprint through actions like using public transportation, conserving water and supporting policies aimed at environmental protection.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4875\n", + "T-Statistic: -6.825454528992923\n", + "P-Value: 0.006434193533715181\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7368421052631579\n", + "Average of Other Ratios: 0.4992690058479532\n", + "T-Statistic: -4.1686084498315\n", + "P-Value: 0.02512588315462554\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 2\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 3\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Done 1 th word\n", + "Done 6 th word\n", + "Done 11 th word\n", + "Done 16 th word\n", + "Done 21 th word\n", + "Done 26 th word\n", + "Done 31 th word\n", + "Done 36 th word\n", + "Done 41 th word\n", + "Done 46 th word\n", + "Done 51 th word\n", + "Done 56 th word\n", + "Done 61 th word\n", + "Done 66 th word\n", + "Done 71 th word\n", + "Done 76 th word\n", + "Done 81 th word\n", + "Done 86 th word\n", + "Done 91 th word\n", + "Original Text:\n", + "The field of biotechnology is revolutionizing medicine and agriculture. Advances in genetic engineering have enabled scientists to develop crops that are resistant to pests and diseases, as well as produce higher yields. In medicine, biotechnology is being used to create personalized treatments based on an individual's genetic makeup. This approach, known as precision medicine, aims to provide more effective and targeted therapies for various diseases. However, the rapid pace of biotechnological innovation also raises ethical and regulatory questions. It is crucial to balance the benefits of these technologies with the potential risks and ensure that they are used responsibly.\n", + "Watermark Areas:\n", + "The field of biotechnology [MASK] revolutionizing medicine and agriculture. [MASK] in genetic engineering have [MASK] scientists to develop crops [MASK] are resistant to pests [MASK] diseases, as well as [MASK] higher yields. In medicine, [MASK] is being used to [MASK] personalized treatments based on [MASK] individual's genetic makeup. This [MASK] known as precision medicine, [MASK] to provide more effective [MASK] targeted therapies for various [MASK] However, the rapid pace [MASK] biotechnological innovation also raises [MASK] and regulatory questions. It [MASK] crucial to balance the [MASK] of these technologies with [MASK] potential risks and ensure [MASK] they are used responsibly.\n", + "Watermarked Text:\n", + "The field of biotechnology is revolutionizing medicine and agriculture. advances in genetic engineering have allowed scientists to develop crops that are resistant to pests and diseases, as well as achieve higher yields. In medicine, biotechnology is being used to develop personalized treatments based on an individual's genetic makeup. This is known as precision medicine, designed to provide more effective and targeted therapies for various diseases However, the rapid pace of biotechnological innovation also raises ethical and regulatory questions. It is crucial to balance the benefits of these technologies with the potential risks and ensure that they are used responsibly.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "C:\\Users\\rrath\\.conda\\envs\\py310\\lib\\site-packages\\scipy\\stats\\_axis_nan_policy.py:523: RuntimeWarning: Precision loss occurred in moment calculation due to catastrophic cancellation. This occurs when the data are nearly identical. Results may be unreliable.\n", + " res = hypotest_fun_out(*samples, **kwds)\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.5\n", + "T-Statistic: -inf\n", + "P-Value: 0.0\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.631578947368421\n", + "Average of Other Ratios: 0.47368421052631576\n", + "T-Statistic: -2.5980762113533156\n", + "P-Value: 0.12168993434632014\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 3\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 4\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Done 1 th word\n", + "Done 6 th word\n", + "Done 11 th word\n", + "Done 16 th word\n", + "Done 21 th word\n", + "Done 26 th word\n", + "Done 31 th word\n", + "Done 36 th word\n", + "Done 41 th word\n", + "Done 46 th word\n", + "Done 51 th word\n", + "Done 56 th word\n", + "Done 61 th word\n", + "Done 66 th word\n", + "Done 71 th word\n", + "Done 76 th word\n", + "Done 81 th word\n", + "Done 86 th word\n", + "Done 91 th word\n", + "Original Text:\n", + "Quantum computing is poised to revolutionize the world of computing. Unlike classical computers, which use bits to represent data as 0s and 1s, quantum computers use qubits, which can exist in multiple states simultaneously. This allows quantum computers to perform complex calculations much faster than their classical counterparts. Potential applications of quantum computing include cryptography, drug discovery, and optimization problems. However, building a practical and scalable quantum computer remains a significant challenge. Researchers are exploring various approaches, such as superconducting qubits and trapped ions, to overcome these hurdles and bring quantum computing closer to reality.\n", + "Watermark Areas:\n", + "Quantum computing is poised [MASK] revolutionize the world of [MASK] Unlike classical computers, which [MASK] bits to represent data [MASK] 0s and 1s, quantum [MASK] use qubits, which can [MASK] in multiple states simultaneously. [MASK] allows quantum computers to [MASK] complex calculations much faster [MASK] their classical counterparts. Potential [MASK] of quantum computing include [MASK] drug discovery, and optimization [MASK] However, building a practical [MASK] scalable quantum computer remains [MASK] significant challenge. Researchers are [MASK] various approaches, such as [MASK] qubits and trapped ions, [MASK] overcome these hurdles and [MASK] quantum computing closer to [MASK]\n", + "Watermarked Text:\n", + "Quantum computing is poised to revolutionize the world of computing Unlike classical computers, which use bits to represent data between 0s and 1s, quantum computers use qubits, which can exist in multiple states simultaneously. this allows quantum computers to perform complex calculations much faster than their classical counterparts. Potential applications of quantum computing include : drug discovery, and optimization . However, building a practical and scalable quantum computer remains a significant challenge. Researchers are exploring various approaches, such as trapped qubits and trapped ions, to overcome these hurdles and bring quantum computing closer to .\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5789473684210527\n", + "Average of Other Ratios: 0.46578947368421053\n", + "T-Statistic: -14.333333333333357\n", + "P-Value: 0.004832243042167172\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.5051169590643274\n", + "T-Statistic: -3.25528426992502\n", + "P-Value: 0.047299956469803\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 4\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 5\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Done 1 th word\n", + "Done 6 th word\n", + "Done 11 th word\n", + "Done 16 th word\n", + "Done 21 th word\n", + "Done 26 th word\n", + "Done 31 th word\n", + "Done 36 th word\n", + "Done 41 th word\n", + "Done 46 th word\n", + "Done 51 th word\n", + "Done 56 th word\n", + "Done 61 th word\n", + "Done 66 th word\n", + "Done 71 th word\n", + "Done 76 th word\n", + "Done 81 th word\n", + "Done 86 th word\n", + "Done 91 th word\n", + "Done 96 th word\n", + "Done 101 th word\n", + "Done 106 th word\n", + "Original Text:\n", + "The internet of things (IoT) is transforming the way we interact with the world around us. IoT refers to the network of interconnected devices that collect and exchange data. These devices range from smart home appliances to industrial sensors, and their applications are vast. In the home, IoT devices can automate tasks like adjusting the thermostat, turning off lights, and monitoring security systems. In industry, IoT is used to optimize supply chains, monitor equipment health, and improve safety. However, the proliferation of IoT devices also raises concerns about security and privacy. Ensuring that these devices are secure and that data is protected is essential for the continued growth of IoT.\n", + "Watermark Areas:\n", + "The internet of things [MASK] is transforming the way [MASK] interact with the world [MASK] us. IoT refers to [MASK] network of interconnected devices [MASK] collect and exchange data. [MASK] devices range from smart [MASK] appliances to industrial sensors, [MASK] their applications are vast. [MASK] the home, IoT devices [MASK] automate tasks like adjusting [MASK] thermostat, turning off lights, [MASK] monitoring security systems. In [MASK] IoT is used to [MASK] supply chains, monitor equipment [MASK] and improve safety. However, [MASK] proliferation of IoT devices [MASK] raises concerns about security [MASK] privacy. Ensuring that these [MASK] are secure and that [MASK] is protected is essential [MASK] the continued growth of [MASK]\n", + "Watermarked Text:\n", + "The internet of things , is transforming the way we interact with the world around us. IoT refers to a network of interconnected devices that collect and exchange data. these devices range from smart home appliances to industrial sensors, and their applications are vast. in the home, IoT devices can automate tasks like adjusting the thermostat, turning off lights, and monitoring security systems. In business IoT is used to monitor supply chains, monitor equipment , and improve safety. However, the proliferation of IoT devices also raises concerns about security and privacy. Ensuring that these devices are secure and that everyone is protected is essential to the continued growth of .\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6956521739130435\n", + "Average of Other Ratios: 0.4204545454545455\n", + "T-Statistic: -4.604144446201748\n", + "P-Value: 0.019265544435432887\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5454545454545454\n", + "Average of Other Ratios: 0.44047619047619047\n", + "T-Statistic: -1.9884454013529376\n", + "P-Value: 0.1408964116764383\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 5\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 6\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Done 1 th word\n", + "Done 6 th word\n", + "Done 11 th word\n", + "Done 16 th word\n", + "Done 21 th word\n", + "Done 26 th word\n", + "Done 31 th word\n", + "Done 36 th word\n", + "Done 41 th word\n", + "Done 46 th word\n", + "Done 51 th word\n", + "Done 56 th word\n", + "Done 61 th word\n", + "Done 66 th word\n", + "Done 71 th word\n", + "Done 76 th word\n", + "Done 81 th word\n", + "Done 86 th word\n", + "Done 91 th word\n", + "Done 96 th word\n", + "Done 101 th word\n", + "Done 106 th word\n", + "Original Text:\n", + "Renewable energy is gaining momentum as a viable solution to the world's energy needs. Solar, wind, and hydro power are among the most common forms of renewable energy, and they offer a sustainable alternative to fossil fuels. Solar power harnesses energy from the sun using photovoltaic cells, while wind power generates electricity through turbines. Hydropower uses the energy of flowing water to produce electricity. These technologies are being adopted at an increasing rate as countries seek to reduce their carbon emissions and transition to cleaner energy sources. The growth of renewable energy is not without challenges, including the need for improved energy storage solutions and the integration of these technologies into existing power grids.\n", + "Watermark Areas:\n", + "Renewable energy is gaining [MASK] as a viable solution [MASK] the world's energy needs. [MASK] wind, and hydro power [MASK] among the most common [MASK] of renewable energy, and [MASK] offer a sustainable alternative [MASK] fossil fuels. Solar power [MASK] energy from the sun [MASK] photovoltaic cells, while wind [MASK] generates electricity through turbines. [MASK] uses the energy of [MASK] water to produce electricity. [MASK] technologies are being adopted [MASK] an increasing rate as [MASK] seek to reduce their [MASK] emissions and transition to [MASK] energy sources. The growth [MASK] renewable energy is not [MASK] challenges, including the need [MASK] improved energy storage solutions [MASK] the integration of these [MASK] into existing power grids.\n", + "Watermarked Text:\n", + "Renewable energy is gaining recognition as a viable solution to the world's energy needs. solar wind, and hydro power are among the most common forms of renewable energy, and they offer a sustainable alternative to fossil fuels. Solar power generates energy from the sun through photovoltaic cells, while wind power generates electricity through turbines. hydro uses the energy of drinking water to produce electricity. new technologies are being adopted at an increasing rate as countries seek to reduce their carbon emissions and transition to renewable energy sources. The growth of renewable energy is not without challenges, including the need for improved energy storage solutions and the integration of these technologies into existing power grids.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.782608695652174\n", + "Average of Other Ratios: 0.4891304347826087\n", + "T-Statistic: -5.7132994884546475\n", + "P-Value: 0.010638289400887542\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7272727272727273\n", + "Average of Other Ratios: 0.5454545454545454\n", + "T-Statistic: -2.529822128134705\n", + "P-Value: 0.08543743614799877\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 6\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 7\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Done 1 th word\n", + "Done 6 th word\n", + "Done 11 th word\n", + "Done 16 th word\n", + "Done 21 th word\n", + "Done 26 th word\n", + "Done 31 th word\n", + "Done 36 th word\n", + "Done 41 th word\n", + "Done 46 th word\n", + "Done 51 th word\n", + "Done 56 th word\n", + "Done 61 th word\n", + "Done 66 th word\n", + "Done 71 th word\n", + "Done 76 th word\n", + "Done 81 th word\n", + "Done 86 th word\n", + "Done 91 th word\n", + "Done 96 th word\n", + "Done 101 th word\n", + "Done 106 th word\n", + "Original Text:\n", + "The rise of e-commerce has transformed the retail industry. Online shopping has become increasingly popular, offering consumers convenience and a wide range of products at their fingertips. Major e-commerce platforms like Amazon, Alibaba, and eBay have disrupted traditional brick-and-mortar stores, leading to significant changes in consumer behavior. The COVID-19 pandemic further accelerated the shift to online shopping, as lockdowns and social distancing measures limited in-person shopping. While e-commerce offers many benefits, it also presents challenges, such as the need for efficient logistics and concerns about data privacy. As the industry continues to evolve, companies are exploring new technologies like augmented reality and artificial intelligence to enhance the online shopping experience.\n", + "Watermark Areas:\n", + "The rise of e-commerce [MASK] transformed the retail industry. [MASK] shopping has become increasingly [MASK] offering consumers convenience and [MASK] wide range of products [MASK] their fingertips. Major e-commerce [MASK] like Amazon, Alibaba, and [MASK] have disrupted traditional brick-and-mortar [MASK] leading to significant changes [MASK] consumer behavior. The COVID-19 [MASK] further accelerated the shift [MASK] online shopping, as lockdowns [MASK] social distancing measures limited [MASK] shopping. While e-commerce offers [MASK] benefits, it also presents [MASK] such as the need [MASK] efficient logistics and concerns [MASK] data privacy. As the [MASK] continues to evolve, companies [MASK] exploring new technologies like [MASK] reality and artificial intelligence [MASK] enhance the online shopping [MASK]\n", + "Watermarked Text:\n", + "The rise of e-commerce has transformed the retail industry. online shopping has become increasingly popular offering consumers convenience and a wide range of products at their fingertips. Major e-commerce companies like Amazon, Alibaba, and others have disrupted traditional brick-and-mortar shopping leading to significant changes in consumer behavior. The COVID-19 has further accelerated the shift towards online shopping, as lockdowns and social distancing measures limited online shopping. While e-commerce offers many benefits, it also presents challenges such as the need for efficient logistics and concerns about data privacy. As the internet continues to evolve, companies are exploring new technologies like augmented reality and artificial intelligence to enhance the online shopping .\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5454545454545454\n", + "Average of Other Ratios: 0.4599802371541502\n", + "T-Statistic: -2.439848527409759\n", + "P-Value: 0.0925127409364643\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "ename": "KeyboardInterrupt", + "evalue": "", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)", + "Cell \u001b[1;32mIn[14], line 24\u001b[0m\n\u001b[0;32m 21\u001b[0m \u001b[38;5;66;03m# print(match_ratios)\u001b[39;00m\n\u001b[0;32m 22\u001b[0m list_of_significance_watermarked\u001b[38;5;241m.\u001b[39mappend(check_significant_difference(match_ratios))\n\u001b[1;32m---> 24\u001b[0m match_ratios \u001b[38;5;241m=\u001b[39m \u001b[43mwatermark_text_and_calculate_matches\u001b[49m\u001b[43m(\u001b[49m\u001b[43mtext\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mmax_offset\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;241;43m5\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[0;32m 25\u001b[0m list_of_significance\u001b[38;5;241m.\u001b[39mappend(check_significant_difference(match_ratios))\n\u001b[0;32m 27\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m___________________________________________________________________________________________________________________________\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n", + "Cell \u001b[1;32mIn[6], line 32\u001b[0m, in \u001b[0;36mwatermark_text_and_calculate_matches\u001b[1;34m(text, model_name, max_offset)\u001b[0m\n\u001b[0;32m 30\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m[MASK]\u001b[39m\u001b[38;5;124m'\u001b[39m \u001b[38;5;129;01min\u001b[39;00m chunk:\n\u001b[0;32m 31\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m---> 32\u001b[0m tempd \u001b[38;5;241m=\u001b[39m \u001b[43mclassifier\u001b[49m\u001b[43m(\u001b[49m\u001b[43mchunk\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 33\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[0;32m 34\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mError processing chunk \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mchunk\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00me\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m)\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\transformers\\pipelines\\fill_mask.py:270\u001b[0m, in \u001b[0;36mFillMaskPipeline.__call__\u001b[1;34m(self, inputs, *args, **kwargs)\u001b[0m\n\u001b[0;32m 248\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m__call__\u001b[39m(\u001b[38;5;28mself\u001b[39m, inputs, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs):\n\u001b[0;32m 249\u001b[0m \u001b[38;5;250m \u001b[39m\u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[0;32m 250\u001b[0m \u001b[38;5;124;03m Fill the masked token in the text(s) given as inputs.\u001b[39;00m\n\u001b[0;32m 251\u001b[0m \n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 268\u001b[0m \u001b[38;5;124;03m - **token_str** (`str`) -- The predicted token (to replace the masked one).\u001b[39;00m\n\u001b[0;32m 269\u001b[0m \u001b[38;5;124;03m \"\"\"\u001b[39;00m\n\u001b[1;32m--> 270\u001b[0m outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28msuper\u001b[39m()\u001b[38;5;241m.\u001b[39m\u001b[38;5;21m__call__\u001b[39m(inputs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[0;32m 271\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(inputs, \u001b[38;5;28mlist\u001b[39m) \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(inputs) \u001b[38;5;241m==\u001b[39m \u001b[38;5;241m1\u001b[39m:\n\u001b[0;32m 272\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m outputs[\u001b[38;5;241m0\u001b[39m]\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\transformers\\pipelines\\base.py:1243\u001b[0m, in \u001b[0;36mPipeline.__call__\u001b[1;34m(self, inputs, num_workers, batch_size, *args, **kwargs)\u001b[0m\n\u001b[0;32m 1235\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mnext\u001b[39m(\n\u001b[0;32m 1236\u001b[0m \u001b[38;5;28miter\u001b[39m(\n\u001b[0;32m 1237\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mget_iterator(\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 1240\u001b[0m )\n\u001b[0;32m 1241\u001b[0m )\n\u001b[0;32m 1242\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m-> 1243\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrun_single\u001b[49m\u001b[43m(\u001b[49m\u001b[43minputs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mpreprocess_params\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mforward_params\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mpostprocess_params\u001b[49m\u001b[43m)\u001b[49m\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\transformers\\pipelines\\base.py:1250\u001b[0m, in \u001b[0;36mPipeline.run_single\u001b[1;34m(self, inputs, preprocess_params, forward_params, postprocess_params)\u001b[0m\n\u001b[0;32m 1248\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mrun_single\u001b[39m(\u001b[38;5;28mself\u001b[39m, inputs, preprocess_params, forward_params, postprocess_params):\n\u001b[0;32m 1249\u001b[0m model_inputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mpreprocess(inputs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mpreprocess_params)\n\u001b[1;32m-> 1250\u001b[0m model_outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mforward(model_inputs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mforward_params)\n\u001b[0;32m 1251\u001b[0m outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mpostprocess(model_outputs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mpostprocess_params)\n\u001b[0;32m 1252\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m outputs\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\transformers\\pipelines\\base.py:1150\u001b[0m, in \u001b[0;36mPipeline.forward\u001b[1;34m(self, model_inputs, **forward_params)\u001b[0m\n\u001b[0;32m 1148\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m inference_context():\n\u001b[0;32m 1149\u001b[0m model_inputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_ensure_tensor_on_device(model_inputs, device\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mdevice)\n\u001b[1;32m-> 1150\u001b[0m model_outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward(model_inputs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mforward_params)\n\u001b[0;32m 1151\u001b[0m model_outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_ensure_tensor_on_device(model_outputs, device\u001b[38;5;241m=\u001b[39mtorch\u001b[38;5;241m.\u001b[39mdevice(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mcpu\u001b[39m\u001b[38;5;124m\"\u001b[39m))\n\u001b[0;32m 1152\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\transformers\\pipelines\\fill_mask.py:127\u001b[0m, in \u001b[0;36mFillMaskPipeline._forward\u001b[1;34m(self, model_inputs)\u001b[0m\n\u001b[0;32m 126\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_forward\u001b[39m(\u001b[38;5;28mself\u001b[39m, model_inputs):\n\u001b[1;32m--> 127\u001b[0m model_outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mmodel(\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mmodel_inputs)\n\u001b[0;32m 128\u001b[0m model_outputs[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124minput_ids\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m model_inputs[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124minput_ids\u001b[39m\u001b[38;5;124m\"\u001b[39m]\n\u001b[0;32m 129\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m model_outputs\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\torch\\nn\\modules\\module.py:1532\u001b[0m, in \u001b[0;36mModule._wrapped_call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m 1530\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_compiled_call_impl(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs) \u001b[38;5;66;03m# type: ignore[misc]\u001b[39;00m\n\u001b[0;32m 1531\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m-> 1532\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_call_impl(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\torch\\nn\\modules\\module.py:1541\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m 1536\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[0;32m 1537\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[0;32m 1538\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[0;32m 1539\u001b[0m \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[0;32m 1540\u001b[0m \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[1;32m-> 1541\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m forward_call(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[0;32m 1543\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m 1544\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\transformers\\models\\bert\\modeling_bert.py:1487\u001b[0m, in \u001b[0;36mBertForMaskedLM.forward\u001b[1;34m(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask, labels, output_attentions, output_hidden_states, return_dict)\u001b[0m\n\u001b[0;32m 1478\u001b[0m \u001b[38;5;250m\u001b[39m\u001b[38;5;124mr\u001b[39m\u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[0;32m 1479\u001b[0m \u001b[38;5;124;03mlabels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*):\u001b[39;00m\n\u001b[0;32m 1480\u001b[0m \u001b[38;5;124;03m Labels for computing the masked language modeling loss. Indices should be in `[-100, 0, ...,\u001b[39;00m\n\u001b[0;32m 1481\u001b[0m \u001b[38;5;124;03m config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the\u001b[39;00m\n\u001b[0;32m 1482\u001b[0m \u001b[38;5;124;03m loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`\u001b[39;00m\n\u001b[0;32m 1483\u001b[0m \u001b[38;5;124;03m\"\"\"\u001b[39;00m\n\u001b[0;32m 1485\u001b[0m return_dict \u001b[38;5;241m=\u001b[39m return_dict \u001b[38;5;28;01mif\u001b[39;00m return_dict \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;28;01melse\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mconfig\u001b[38;5;241m.\u001b[39muse_return_dict\n\u001b[1;32m-> 1487\u001b[0m outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mbert\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 1488\u001b[0m \u001b[43m \u001b[49m\u001b[43minput_ids\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1489\u001b[0m \u001b[43m \u001b[49m\u001b[43mattention_mask\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mattention_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1490\u001b[0m \u001b[43m \u001b[49m\u001b[43mtoken_type_ids\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtoken_type_ids\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1491\u001b[0m \u001b[43m \u001b[49m\u001b[43mposition_ids\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mposition_ids\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1492\u001b[0m \u001b[43m \u001b[49m\u001b[43mhead_mask\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mhead_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1493\u001b[0m \u001b[43m \u001b[49m\u001b[43minputs_embeds\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43minputs_embeds\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1494\u001b[0m \u001b[43m \u001b[49m\u001b[43mencoder_hidden_states\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mencoder_hidden_states\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1495\u001b[0m \u001b[43m \u001b[49m\u001b[43mencoder_attention_mask\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mencoder_attention_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1496\u001b[0m \u001b[43m \u001b[49m\u001b[43moutput_attentions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moutput_attentions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1497\u001b[0m \u001b[43m \u001b[49m\u001b[43moutput_hidden_states\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moutput_hidden_states\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1498\u001b[0m \u001b[43m \u001b[49m\u001b[43mreturn_dict\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mreturn_dict\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1499\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 1501\u001b[0m sequence_output \u001b[38;5;241m=\u001b[39m outputs[\u001b[38;5;241m0\u001b[39m]\n\u001b[0;32m 1502\u001b[0m prediction_scores \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mcls(sequence_output)\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\torch\\nn\\modules\\module.py:1532\u001b[0m, in \u001b[0;36mModule._wrapped_call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m 1530\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_compiled_call_impl(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs) \u001b[38;5;66;03m# type: ignore[misc]\u001b[39;00m\n\u001b[0;32m 1531\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m-> 1532\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_call_impl(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\torch\\nn\\modules\\module.py:1541\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m 1536\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[0;32m 1537\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[0;32m 1538\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[0;32m 1539\u001b[0m \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[0;32m 1540\u001b[0m \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[1;32m-> 1541\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m forward_call(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[0;32m 1543\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m 1544\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\transformers\\models\\bert\\modeling_bert.py:1137\u001b[0m, in \u001b[0;36mBertModel.forward\u001b[1;34m(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask, past_key_values, use_cache, output_attentions, output_hidden_states, return_dict)\u001b[0m\n\u001b[0;32m 1130\u001b[0m \u001b[38;5;66;03m# Prepare head mask if needed\u001b[39;00m\n\u001b[0;32m 1131\u001b[0m \u001b[38;5;66;03m# 1.0 in head_mask indicate we keep the head\u001b[39;00m\n\u001b[0;32m 1132\u001b[0m \u001b[38;5;66;03m# attention_probs has shape bsz x n_heads x N x N\u001b[39;00m\n\u001b[0;32m 1133\u001b[0m \u001b[38;5;66;03m# input head_mask has shape [num_heads] or [num_hidden_layers x num_heads]\u001b[39;00m\n\u001b[0;32m 1134\u001b[0m \u001b[38;5;66;03m# and head_mask is converted to shape [num_hidden_layers x batch x num_heads x seq_length x seq_length]\u001b[39;00m\n\u001b[0;32m 1135\u001b[0m head_mask \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mget_head_mask(head_mask, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mconfig\u001b[38;5;241m.\u001b[39mnum_hidden_layers)\n\u001b[1;32m-> 1137\u001b[0m encoder_outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mencoder\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 1138\u001b[0m \u001b[43m \u001b[49m\u001b[43membedding_output\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1139\u001b[0m \u001b[43m \u001b[49m\u001b[43mattention_mask\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mextended_attention_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1140\u001b[0m \u001b[43m \u001b[49m\u001b[43mhead_mask\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mhead_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1141\u001b[0m \u001b[43m \u001b[49m\u001b[43mencoder_hidden_states\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mencoder_hidden_states\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1142\u001b[0m \u001b[43m \u001b[49m\u001b[43mencoder_attention_mask\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mencoder_extended_attention_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1143\u001b[0m \u001b[43m \u001b[49m\u001b[43mpast_key_values\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mpast_key_values\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1144\u001b[0m \u001b[43m \u001b[49m\u001b[43muse_cache\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43muse_cache\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1145\u001b[0m \u001b[43m \u001b[49m\u001b[43moutput_attentions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moutput_attentions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1146\u001b[0m \u001b[43m \u001b[49m\u001b[43moutput_hidden_states\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moutput_hidden_states\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1147\u001b[0m \u001b[43m \u001b[49m\u001b[43mreturn_dict\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mreturn_dict\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1148\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 1149\u001b[0m sequence_output \u001b[38;5;241m=\u001b[39m encoder_outputs[\u001b[38;5;241m0\u001b[39m]\n\u001b[0;32m 1150\u001b[0m pooled_output \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mpooler(sequence_output) \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mpooler \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;28;01melse\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\torch\\nn\\modules\\module.py:1532\u001b[0m, in \u001b[0;36mModule._wrapped_call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m 1530\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_compiled_call_impl(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs) \u001b[38;5;66;03m# type: ignore[misc]\u001b[39;00m\n\u001b[0;32m 1531\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m-> 1532\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_call_impl(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\torch\\nn\\modules\\module.py:1541\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m 1536\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[0;32m 1537\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[0;32m 1538\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[0;32m 1539\u001b[0m \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[0;32m 1540\u001b[0m \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[1;32m-> 1541\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m forward_call(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[0;32m 1543\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m 1544\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\transformers\\models\\bert\\modeling_bert.py:690\u001b[0m, in \u001b[0;36mBertEncoder.forward\u001b[1;34m(self, hidden_states, attention_mask, head_mask, encoder_hidden_states, encoder_attention_mask, past_key_values, use_cache, output_attentions, output_hidden_states, return_dict)\u001b[0m\n\u001b[0;32m 679\u001b[0m layer_outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_gradient_checkpointing_func(\n\u001b[0;32m 680\u001b[0m layer_module\u001b[38;5;241m.\u001b[39m\u001b[38;5;21m__call__\u001b[39m,\n\u001b[0;32m 681\u001b[0m hidden_states,\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 687\u001b[0m output_attentions,\n\u001b[0;32m 688\u001b[0m )\n\u001b[0;32m 689\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m--> 690\u001b[0m layer_outputs \u001b[38;5;241m=\u001b[39m \u001b[43mlayer_module\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 691\u001b[0m \u001b[43m \u001b[49m\u001b[43mhidden_states\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 692\u001b[0m \u001b[43m \u001b[49m\u001b[43mattention_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 693\u001b[0m \u001b[43m \u001b[49m\u001b[43mlayer_head_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 694\u001b[0m \u001b[43m \u001b[49m\u001b[43mencoder_hidden_states\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 695\u001b[0m \u001b[43m \u001b[49m\u001b[43mencoder_attention_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 696\u001b[0m \u001b[43m \u001b[49m\u001b[43mpast_key_value\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 697\u001b[0m \u001b[43m \u001b[49m\u001b[43moutput_attentions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 698\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 700\u001b[0m hidden_states \u001b[38;5;241m=\u001b[39m layer_outputs[\u001b[38;5;241m0\u001b[39m]\n\u001b[0;32m 701\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m use_cache:\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\torch\\nn\\modules\\module.py:1532\u001b[0m, in \u001b[0;36mModule._wrapped_call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m 1530\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_compiled_call_impl(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs) \u001b[38;5;66;03m# type: ignore[misc]\u001b[39;00m\n\u001b[0;32m 1531\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m-> 1532\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_call_impl(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\torch\\nn\\modules\\module.py:1541\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m 1536\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[0;32m 1537\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[0;32m 1538\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[0;32m 1539\u001b[0m \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[0;32m 1540\u001b[0m \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[1;32m-> 1541\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m forward_call(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[0;32m 1543\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m 1544\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\transformers\\models\\bert\\modeling_bert.py:580\u001b[0m, in \u001b[0;36mBertLayer.forward\u001b[1;34m(self, hidden_states, attention_mask, head_mask, encoder_hidden_states, encoder_attention_mask, past_key_value, output_attentions)\u001b[0m\n\u001b[0;32m 568\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mforward\u001b[39m(\n\u001b[0;32m 569\u001b[0m \u001b[38;5;28mself\u001b[39m,\n\u001b[0;32m 570\u001b[0m hidden_states: torch\u001b[38;5;241m.\u001b[39mTensor,\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 577\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m Tuple[torch\u001b[38;5;241m.\u001b[39mTensor]:\n\u001b[0;32m 578\u001b[0m \u001b[38;5;66;03m# decoder uni-directional self-attention cached key/values tuple is at positions 1,2\u001b[39;00m\n\u001b[0;32m 579\u001b[0m self_attn_past_key_value \u001b[38;5;241m=\u001b[39m past_key_value[:\u001b[38;5;241m2\u001b[39m] \u001b[38;5;28;01mif\u001b[39;00m past_key_value \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;28;01melse\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m--> 580\u001b[0m self_attention_outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mattention\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 581\u001b[0m \u001b[43m \u001b[49m\u001b[43mhidden_states\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 582\u001b[0m \u001b[43m \u001b[49m\u001b[43mattention_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 583\u001b[0m \u001b[43m \u001b[49m\u001b[43mhead_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 584\u001b[0m \u001b[43m \u001b[49m\u001b[43moutput_attentions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moutput_attentions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 585\u001b[0m \u001b[43m \u001b[49m\u001b[43mpast_key_value\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mself_attn_past_key_value\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 586\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 587\u001b[0m attention_output \u001b[38;5;241m=\u001b[39m self_attention_outputs[\u001b[38;5;241m0\u001b[39m]\n\u001b[0;32m 589\u001b[0m \u001b[38;5;66;03m# if decoder, the last output is tuple of self-attn cache\u001b[39;00m\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\torch\\nn\\modules\\module.py:1532\u001b[0m, in \u001b[0;36mModule._wrapped_call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m 1530\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_compiled_call_impl(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs) \u001b[38;5;66;03m# type: ignore[misc]\u001b[39;00m\n\u001b[0;32m 1531\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m-> 1532\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_call_impl(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\torch\\nn\\modules\\module.py:1541\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m 1536\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[0;32m 1537\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[0;32m 1538\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[0;32m 1539\u001b[0m \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[0;32m 1540\u001b[0m \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[1;32m-> 1541\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m forward_call(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[0;32m 1543\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m 1544\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\transformers\\models\\bert\\modeling_bert.py:510\u001b[0m, in \u001b[0;36mBertAttention.forward\u001b[1;34m(self, hidden_states, attention_mask, head_mask, encoder_hidden_states, encoder_attention_mask, past_key_value, output_attentions)\u001b[0m\n\u001b[0;32m 500\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mforward\u001b[39m(\n\u001b[0;32m 501\u001b[0m \u001b[38;5;28mself\u001b[39m,\n\u001b[0;32m 502\u001b[0m hidden_states: torch\u001b[38;5;241m.\u001b[39mTensor,\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 508\u001b[0m output_attentions: Optional[\u001b[38;5;28mbool\u001b[39m] \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mFalse\u001b[39;00m,\n\u001b[0;32m 509\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m Tuple[torch\u001b[38;5;241m.\u001b[39mTensor]:\n\u001b[1;32m--> 510\u001b[0m self_outputs \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mself\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 511\u001b[0m \u001b[43m \u001b[49m\u001b[43mhidden_states\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 512\u001b[0m \u001b[43m \u001b[49m\u001b[43mattention_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 513\u001b[0m \u001b[43m \u001b[49m\u001b[43mhead_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 514\u001b[0m \u001b[43m \u001b[49m\u001b[43mencoder_hidden_states\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 515\u001b[0m \u001b[43m \u001b[49m\u001b[43mencoder_attention_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 516\u001b[0m \u001b[43m \u001b[49m\u001b[43mpast_key_value\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 517\u001b[0m \u001b[43m \u001b[49m\u001b[43moutput_attentions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 518\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 519\u001b[0m attention_output \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39moutput(self_outputs[\u001b[38;5;241m0\u001b[39m], hidden_states)\n\u001b[0;32m 520\u001b[0m outputs \u001b[38;5;241m=\u001b[39m (attention_output,) \u001b[38;5;241m+\u001b[39m self_outputs[\u001b[38;5;241m1\u001b[39m:] \u001b[38;5;66;03m# add attentions if we output them\u001b[39;00m\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\torch\\nn\\modules\\module.py:1532\u001b[0m, in \u001b[0;36mModule._wrapped_call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m 1530\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_compiled_call_impl(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs) \u001b[38;5;66;03m# type: ignore[misc]\u001b[39;00m\n\u001b[0;32m 1531\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m-> 1532\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_call_impl(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\torch\\nn\\modules\\module.py:1541\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m 1536\u001b[0m \u001b[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[0;32m 1537\u001b[0m \u001b[38;5;66;03m# this function, and just call forward.\u001b[39;00m\n\u001b[0;32m 1538\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_forward_pre_hooks\n\u001b[0;32m 1539\u001b[0m \u001b[38;5;129;01mor\u001b[39;00m _global_backward_pre_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_backward_hooks\n\u001b[0;32m 1540\u001b[0m \u001b[38;5;129;01mor\u001b[39;00m _global_forward_hooks \u001b[38;5;129;01mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[1;32m-> 1541\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m forward_call(\u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[0;32m 1543\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m 1544\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\transformers\\models\\bert\\modeling_bert.py:435\u001b[0m, in \u001b[0;36mBertSdpaSelfAttention.forward\u001b[1;34m(self, hidden_states, attention_mask, head_mask, encoder_hidden_states, encoder_attention_mask, past_key_value, output_attentions)\u001b[0m\n\u001b[0;32m 431\u001b[0m \u001b[38;5;66;03m# The tgt_len > 1 is necessary to match with AttentionMaskConverter.to_causal_4d that does not create a causal\u001b[39;00m\n\u001b[0;32m 432\u001b[0m \u001b[38;5;66;03m# mask in case tgt_len == 1.\u001b[39;00m\n\u001b[0;32m 433\u001b[0m is_causal \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mis_decoder \u001b[38;5;129;01mand\u001b[39;00m attention_mask \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;129;01mand\u001b[39;00m tgt_len \u001b[38;5;241m>\u001b[39m \u001b[38;5;241m1\u001b[39m\n\u001b[1;32m--> 435\u001b[0m attn_output \u001b[38;5;241m=\u001b[39m \u001b[43mtorch\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mnn\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfunctional\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mscaled_dot_product_attention\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 436\u001b[0m \u001b[43m \u001b[49m\u001b[43mquery_layer\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 437\u001b[0m \u001b[43m \u001b[49m\u001b[43mkey_layer\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 438\u001b[0m \u001b[43m \u001b[49m\u001b[43mvalue_layer\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 439\u001b[0m \u001b[43m \u001b[49m\u001b[43mattn_mask\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mattention_mask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 440\u001b[0m \u001b[43m \u001b[49m\u001b[43mdropout_p\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mdropout_prob\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mif\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mtraining\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01melse\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;241;43m0.0\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[0;32m 441\u001b[0m \u001b[43m \u001b[49m\u001b[43mis_causal\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mis_causal\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 442\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 444\u001b[0m attn_output \u001b[38;5;241m=\u001b[39m attn_output\u001b[38;5;241m.\u001b[39mtranspose(\u001b[38;5;241m1\u001b[39m, \u001b[38;5;241m2\u001b[39m)\n\u001b[0;32m 445\u001b[0m attn_output \u001b[38;5;241m=\u001b[39m attn_output\u001b[38;5;241m.\u001b[39mreshape(bsz, tgt_len, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mall_head_size)\n", + "\u001b[1;31mKeyboardInterrupt\u001b[0m: " + ] + } + ], + "source": [ + "list_of_significance = []\n", + "list_of_significance_watermarked = []\n", + "count_t = 0\n", + "for text in texts:\n", + " count_t+=1\n", + " print(\"___________________________________________________________________________________________________________________________\")\n", + " print(\"Doing\", count_t)\n", + " print(\"___________________________________________________________________________________________________________________________\")\n", + "\n", + " words_to_add = [\"example\", \"test\", \"random\", \"insert\"]\n", + " num_words_to_add = 5\n", + "\n", + " # modified_text = randomly_add_words(text, words_to_add, num_words_to_add)\n", + " modified_text = randomly_add_words(watermark_text(text, offset=0), words_to_add, num_words_to_add)\n", + " # print(\"Original Text:\")\n", + " # print(text)\n", + " # print(\"\\nModified Text:\")\n", + " # print(modified_text)\n", + "\n", + " match_ratios = watermark_text_and_calculate_matches(modified_text, max_offset=5)\n", + " # print(match_ratios)\n", + " list_of_significance_watermarked.append(check_significant_difference(match_ratios))\n", + "\n", + " match_ratios = watermark_text_and_calculate_matches(text, max_offset=5)\n", + " list_of_significance.append(check_significant_difference(match_ratios))\n", + "\n", + " print(\"___________________________________________________________________________________________________________________________\")\n", + " print(\"Done\", count_t, )\n", + " print(\"___________________________________________________________________________________________________________________________\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "TQd_XP_kRZ3r", + "outputId": "4f585fea-470c-4f09-8397-182c269af1ee" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[[0.5789473684210527, 0.4375, -11.315789473684214, 0.0014802633181549077],\n", + " [0.7368421052631579,\n", + " 0.4992690058479532,\n", + " -4.1686084498315,\n", + " 0.02512588315462554],\n", + " [0.631578947368421,\n", + " 0.47368421052631576,\n", + " -2.5980762113533156,\n", + " 0.12168993434632014],\n", + " [0.6666666666666666,\n", + " 0.5051169590643274,\n", + " -3.25528426992502,\n", + " 0.047299956469803],\n", + " [0.5454545454545454,\n", + " 0.44047619047619047,\n", + " -1.9884454013529376,\n", + " 0.1408964116764383],\n", + " [0.7272727272727273,\n", + " 0.5454545454545454,\n", + " -2.529822128134705,\n", + " 0.08543743614799877],\n", + " [0.5, 0.4285714285714286, -3.674234614174766, 0.034896984510150934],\n", + " [0.45, 0.36140350877192984, -1.789925042646048, 0.21535497619213528],\n", + " [0.6363636363636364,\n", + " 0.5524891774891776,\n", + " -4.925394256602069,\n", + " 0.01603915968463389],\n", + " [0.6363636363636364,\n", + " 0.5004940711462451,\n", + " -2.8968775241076448,\n", + " 0.0626611732957653],\n", + " [0.75, 0.5776315789473684, -4.168368422873468, 0.02512970789136552],\n", + " [0.6818181818181818,\n", + " 0.5568181818181819,\n", + " -3.666666666666662,\n", + " 0.03508151471548204],\n", + " [0.7391304347826086,\n", + " 0.42984189723320154,\n", + " -7.289590560310877,\n", + " 0.005329596912408047],\n", + " [0.782608695652174,\n", + " 0.4936594202898551,\n", + " -7.972508980104777,\n", + " 0.004117361652430399],\n", + " [0.6363636363636364,\n", + " 0.5113636363636364,\n", + " -3.22047024073016,\n", + " 0.04856685655980099],\n", + " [0.6521739130434783,\n", + " 0.5434782608695652,\n", + " -2.3797114365109158,\n", + " 0.09764327274027122],\n", + " [0.631578947368421,\n", + " 0.47368421052631576,\n", + " -2.5980762113533156,\n", + " 0.12168993434632014],\n", + " [0.7142857142857143,\n", + " 0.5367965367965368,\n", + " -2.3442928638434024,\n", + " 0.10082728660926546],\n", + " [0.47619047619047616,\n", + " 0.38095238095238093,\n", + " -3.4641016151377544,\n", + " 0.07417990022744853],\n", + " [0.5, 0.4285714285714286, -3.674234614174766, 0.034896984510150934],\n", + " [0.5454545454545454,\n", + " 0.4631093544137022,\n", + " -4.2649449620933755,\n", + " 0.05082148124684452],\n", + " [0.782608695652174,\n", + " 0.4936594202898551,\n", + " -7.972508980104777,\n", + " 0.004117361652430399],\n", + " [0.6363636363636364,\n", + " 0.5113636363636364,\n", + " -3.22047024073016,\n", + " 0.04856685655980099],\n", + " [0.6521739130434783,\n", + " 0.5434782608695652,\n", + " -2.3797114365109158,\n", + " 0.09764327274027122],\n", + " [0.631578947368421,\n", + " 0.47368421052631576,\n", + " -2.5980762113533156,\n", + " 0.12168993434632014],\n", + " [0.7142857142857143,\n", + " 0.5367965367965368,\n", + " -2.3442928638434024,\n", + " 0.10082728660926546],\n", + " [0.47619047619047616,\n", + " 0.38095238095238093,\n", + " -3.4641016151377544,\n", + " 0.07417990022744853],\n", + " [0.5, 0.4285714285714286, -3.674234614174766, 0.034896984510150934],\n", + " [0.5454545454545454,\n", + " 0.4631093544137022,\n", + " -4.2649449620933755,\n", + " 0.05082148124684452]]" + ] + }, + "execution_count": 47, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "gm9oCHWlSi4I", + "outputId": "c192b477-1e37-4336-f126-2f77e44eb6f6" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Ratio Average Others T-Statistic P-Value || Highest Ratio Average Others T-Statistic P-Value \n", + "0.5789473684210527 0.4375 -11.315789473684214 0.0014802633181549077 || 0.65 0.41666666666666663 -5.17705132919467 0.013988180239752648\n", + "0.7368421052631579 0.4992690058479532 -4.1686084498315 0.02512588315462554 || 0.631578947368421 0.5072368421052631 -3.7039840906304633 0.034183520845761046\n", + "0.631578947368421 0.47368421052631576 -2.5980762113533156 0.12168993434632014 || 0.8 0.44999999999999996 -9.899494936611665 0.002192318898657741\n", + "0.6666666666666666 0.5051169590643274 -3.25528426992502 0.047299956469803 || 0.631578947368421 0.5052631578947369 -3.3070695276573017 0.04549183755402306 \n", + "0.5454545454545454 0.44047619047619047 -1.9884454013529376 0.1408964116764383 || 0.6363636363636364 0.3932806324110672 -5.671556095740365 0.010858631561421467\n", + "0.7272727272727273 0.5454545454545454 -2.529822128134705 0.08543743614799877 || 0.782608695652174 0.5 -3.0929011843007626 0.0535919356301439 \n", + "0.5 0.4285714285714286 -3.674234614174766 0.034896984510150934 || 0.5454545454545454 0.39377470355731226 -3.7778595133554176 0.032490871457917674\n", + "0.45 0.36140350877192984 -1.789925042646048 0.21535497619213528 || 0.5714285714285714 0.407936507936508 -2.525754294555235 0.12746322930311096 \n", + "0.6363636363636364 0.5524891774891776 -4.925394256602069 0.01603915968463389 || 0.7272727272727273 0.5093873517786561 -4.676780667650381 0.018467037431746196\n", + "0.6363636363636364 0.5004940711462451 -2.8968775241076448 0.0626611732957653 || 0.7083333333333334 0.5040760869565217 -3.3948179538648735 0.042623438183825496\n", + "0.75 0.5776315789473684 -4.168368422873468 0.02512970789136552 || 0.6190476190476191 0.5369047619047619 -3.7736294416002862 0.03258485403885965 \n", + "0.6818181818181818 0.5568181818181819 -3.666666666666662 0.03508151471548204 || 0.6956521739130435 0.532608695652174 -3.382407126012729 0.043014906734981546\n", + "0.7391304347826086 0.42984189723320154 -7.289590560310877 0.005329596912408047 || 0.7083333333333334 0.4433876811594203 -4.490136077665652 0.02061159932091642 \n", + "0.782608695652174 0.4936594202898551 -7.972508980104777 0.004117361652430399 || 0.6666666666666666 0.5354166666666667 -5.5468407098514305 0.011553575011403559\n", + "0.6363636363636364 0.5113636363636364 -3.22047024073016 0.04856685655980099 || 0.6086956521739131 0.47826086956521735 -4.242640687119289 0.023981199790656615\n", + "0.6521739130434783 0.5434782608695652 -2.3797114365109158 0.09764327274027122 || 0.8333333333333334 0.46557971014492755 -4.234837745291732 0.02409863068609194 \n", + "0.631578947368421 0.47368421052631576 -2.5980762113533156 0.12168993434632014 || 0.6 0.48333333333333334 -3.4999999999999987 0.07282735005446936 \n", + "0.7142857142857143 0.5367965367965368 -2.3442928638434024 0.10082728660926546 || 0.782608695652174 0.549901185770751 -4.594812178568088 0.01937136230021868 \n", + "0.47619047619047616 0.38095238095238093 -3.4641016151377544 0.07417990022744853 || 0.6363636363636364 0.4090909090909091 -4.08248290463863 0.026547885467199484\n", + "0.5 0.4285714285714286 -3.674234614174766 0.034896984510150934 || 0.7272727272727273 0.42539525691699603 -4.9868551538544414 0.015503886330756058\n", + "0.5454545454545454 0.4631093544137022 -4.2649449620933755 0.05082148124684452 || 0.6956521739130435 0.44157608695652173 -7.251548965980652 0.0054102533801680865\n", + "0.782608695652174 0.4936594202898551 -7.972508980104777 0.004117361652430399 || 0.625 0.49222222222222217 -3.9180327868852447 0.05939767081769266 \n", + "0.6363636363636364 0.5113636363636364 -3.22047024073016 0.04856685655980099 || 0.5217391304347826 0.44565217391304346 -3.6556307750696546 0.03535284700251738 \n", + "0.6521739130434783 0.5434782608695652 -2.3797114365109158 0.09764327274027122 || 0.6666666666666666 0.5 -13.279056191361398 0.005623287315631082\n", + "0.631578947368421 0.47368421052631576 -2.5980762113533156 0.12168993434632014 || 0.7 0.475 -8.999999999999995 0.0028958121618641495\n", + "0.7142857142857143 0.5367965367965368 -2.3442928638434024 0.10082728660926546 || 0.7391304347826086 0.5602766798418972 -3.356266857779692 0.04385449037496923 \n", + "0.47619047619047616 0.38095238095238093 -3.4641016151377544 0.07417990022744853 || 0.5 0.38636363636363635 -2.611164839335468 0.07960498081790623 \n", + "0.5 0.4285714285714286 -3.674234614174766 0.034896984510150934 || 0.6086956521739131 0.44318181818181823 -4.855072463768116 0.01668150816820796 \n", + "0.5454545454545454 0.4631093544137022 -4.2649449620933755 0.05082148124684452 || 0.6666666666666666 0.4673913043478261 -2.77438299767925 0.0693145043773778 \n" + ] + } + ], + "source": [ + "print(f\"{'Highest Ratio':<20} {'Average Others':<20} {'T-Statistic':<20} {'P-Value':<20} || {'Highest Ratio':<20} {'Average Others':<20} {'T-Statistic':<20} {'P-Value':<20}\")\n", + "\n", + "# Print each pair of lists side by side\n", + "for sig, wm_sig in zip(list_of_significance, list_of_significance_watermarked):\n", + " print(f\"{sig[0]:<20} {sig[1]:<20} {sig[2]:<20} {sig[3]:<20} || {wm_sig[0]:<20} {wm_sig[1]:<20} {wm_sig[2]:<20} {wm_sig[3]:<20}\")" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000 + }, + "id": "brgJ75fzTzry", + "outputId": "59e7ac15-cb87-4abc-b421-144e302e6b1b" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "First few rows of the DataFrame:\n", + " Highest Ratio Average Others T-Statistic P-Value Label\n", + "0 0.233333 0.182203 -3.532758 0.038563 Original\n", + "1 0.203390 0.139195 -3.440591 0.041218 Original\n", + "2 0.338983 0.270339 -2.228608 0.112142 Original\n", + "3 0.254237 0.168362 -2.451613 0.246559 Original\n", + "4 0.288136 0.210876 -5.467540 0.012026 Original\n", + "\n", + "Statistical Summary:\n", + " Highest Ratio Average Others T-Statistic P-Value\n", + "count 4000.000000 4000.000000 3999.000000 3999.000000\n", + "mean 0.490285 0.339968 -6.076672 0.036783\n", + "std 0.128376 0.082900 5.580957 0.043217\n", + "min 0.101695 0.066667 -111.524590 0.000002\n", + "25% 0.416667 0.296610 -6.938964 0.006418\n", + "50% 0.491525 0.354732 -4.431515 0.021973\n", + "75% 0.573770 0.398224 -3.176861 0.052069\n", + "max 0.868852 0.580601 -1.166065 0.451288\n", + "\n", + "Missing Values:\n", + "Highest Ratio 0\n", + "Average Others 0\n", + "T-Statistic 1\n", + "P-Value 1\n", + "Label 0\n", + "dtype: int64\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA/AAAAK9CAYAAACD7ocIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAADYaklEQVR4nOzdd3hUZd7G8XvSeyeVQELovUiJIBBA6TZ8FQuoy6qrgKKua++Fde0F2+qKjbWtoiK9KlV6b8HQSUKAJKSXOe8fQ0YiNWGSM5N8P9c1V2bOnHnOb0I0uedpFsMwDAEAAAAAAKfmZnYBAAAAAADg3AjwAAAAAAC4AAI8AAAAAAAugAAPAAAAAIALIMADAAAAAOACCPAAAAAAALgAAjwAAAAAAC6AAA8AAAAAgAsgwAMAAAAA4AII8AAAp/fUU0/JYrHUyrX69u2rvn372h8vXLhQFotF3377ba1c/5ZbblFCQkKtXKu68vLy9Ne//lXR0dGyWCyaMGGCw9qePHmyLBaLdu/eXe3Xrlq1ymH1uLrdu3fLYrFo8uTJZpcCAHAAAjwAoFZVhKyKm4+Pj2JjYzVw4EC9+eabOn78uEOuc/DgQT311FNat26dQ9pzJGeu7Xy88MILmjx5su6880599tlnGjVq1BnPTUhI0LBhw077XG1/OFJT3nnnnSoF5JN//i0Wi4KCgtSnTx/9/PPP1a5hypQpev3116v9egCAa/AwuwAAQP30zDPPKDExUaWlpUpPT9fChQs1YcIEvfrqq/rxxx/Vvn17+7mPPfaYHnrooSq1f/DgQT399NNKSEhQx44dz/t1s2fPrtJ1quNstf373/+W1Wqt8RouxPz589WjRw89+eSTDm971KhRGjlypLy9vR3edk155513FBERoVtuueW8X3PppZdq9OjRMgxDe/bs0bvvvqvhw4drxowZGjhwYJVrmDJlijZt2nTKaIjGjRursLBQnp6eVW4TAOB8CPAAAFMMHjxYF110kf3xww8/rPnz52vYsGG6/PLLtXXrVvn6+kqSPDw85OFRs7+yCgoK5OfnJy8vrxq9zrm4QtDKzMxU69ata6Rtd3d3ubu710jbzqR58+a66aab7I9HjBih1q1b64033qhWgD+TilEuAIC6gSH0AACn0a9fPz3++OPas2ePPv/8c/vx082BnzNnjnr16qWQkBAFBASoRYsWeuSRRyTZhmZ37dpVknTrrbfahypXDHPu27ev2rZtq9WrV6t3797y8/Ozv/bPc+ArlJeX65FHHlF0dLT8/f11+eWXa9++fZXOSUhIOG0v7Mltnqu2082Bz8/P1/3336/4+Hh5e3urRYsWevnll2UYRqXzLBaLxo0bp6lTp6pt27by9vZWmzZtNHPmzNN/w/8kMzNTY8aMUVRUlHx8fNShQwd98skn9ucrhrynpaXp559/ttdenfnqZ3K6OfBWq1VPPfWUYmNj5efnp5SUFG3ZsuWM3+/i4mLdd999atCggfz9/XXVVVfp8OHDp5w3Y8YMXXLJJfL391dgYKCGDh2qzZs3VzonPT1dt956qxo2bChvb2/FxMToiiuusNeXkJCgzZs3a9GiRfbvx+l+fs6lVatWioiI0K5duyod/+GHHzR06FDFxsbK29tbSUlJevbZZ1VeXm4/p2/fvvr555+1Z88eew0VP0NnmgM/f/58+3sPCQnRFVdcoa1bt1a5bgBA7aIHHgDgVEaNGqVHHnlEs2fP1m233XbaczZv3qxhw4apffv2euaZZ+Tt7a3U1FQtWbJEki0MPfPMM3riiSd0++2365JLLpEkXXzxxfY2jhw5osGDB2vkyJG66aabFBUVdda6nn/+eVksFj344IPKzMzU66+/rgEDBmjdunX2kQLn43xqO5lhGLr88su1YMECjRkzRh07dtSsWbP0wAMP6MCBA3rttdcqnb948WJ99913uuuuuxQYGKg333xTI0aM0N69exUeHn7GugoLC9W3b1+lpqZq3LhxSkxM1DfffKNbbrlF2dnZuueee9SqVSt99tlnuvfee9WwYUPdf//9kqQGDRqc9T2XlpYqKyvrlOM5OTlnfV2Fhx9+WP/61780fPhwDRw4UOvXr9fAgQNVVFR02vPHjx+v0NBQPfnkk9q9e7def/11jRs3Tl999ZX9nM8++0w333yzBg4cqBdffFEFBQV699131atXL61du9YegEeMGKHNmzdr/PjxSkhIUGZmpubMmaO9e/cqISFBr7/+usaPH6+AgAA9+uijknTOn6XTycnJ0bFjx5SUlFTp+OTJkxUQEKD77rtPAQEBmj9/vp544gnl5ubqpZdekiQ9+uijysnJ0f79++0/DwEBAWe81ty5czV48GA1adJETz31lAoLC/XWW2+pZ8+eWrNmjdMvoggA9ZoBAEAt+vjjjw1JxsqVK894TnBwsNGpUyf74yeffNI4+VfWa6+9ZkgyDh8+fMY2Vq5caUgyPv7441Oe69OnjyHJeO+99077XJ8+feyPFyxYYEgy4uLijNzcXPvxr7/+2pBkvPHGG/ZjjRs3Nm6++eZztnm22m6++WajcePG9sdTp041JBnPPfdcpfOuueYaw2KxGKmpqfZjkgwvL69Kx9avX29IMt56661TrnWy119/3ZBkfP755/ZjJSUlRnJyshEQEFDpvTdu3NgYOnToWds7+VxJZ71988039vMrfj7S0tIMwzCM9PR0w8PDw7jyyisrtfvUU08Zkip9vyteO2DAAMNqtdqP33vvvYa7u7uRnZ1tGIZhHD9+3AgJCTFuu+22Sm2mp6cbwcHB9uPHjh0zJBkvvfTSWd9jmzZtKv37noskY8yYMcbhw4eNzMxMY9WqVcagQYNOe62CgoJTXn/HHXcYfn5+RlFRkf3Y0KFDK/3cVEhLSzvlZ61jx45GZGSkceTIEfux9evXG25ubsbo0aPP+30AAGofQ+gBAE4nICDgrKvRh4SESLINL67ugm/e3t669dZbz/v80aNHKzAw0P74mmuuUUxMjKZPn16t65+v6dOny93dXXfffXel4/fff78Mw9CMGTMqHR8wYEClXtz27dsrKChIv//++zmvEx0dreuvv95+zNPTU3fffbfy8vK0aNGiar+H7t27a86cOafcXn755XO+dt68eSorK9Ndd91V6fj48ePP+Jrbb7+90pSLSy65ROXl5dqzZ48k2/SL7OxsXX/99crKyrLf3N3d1b17dy1YsECS5OvrKy8vLy1cuFDHjh2rzls/o48++kgNGjRQZGSkLrroIs2bN0//+Mc/dN9991U67+TRHcePH1dWVpYuueQSFRQUaNu2bVW+7qFDh7Ru3TrdcsstCgsLsx9v3769Lr300hr/eQYAXBiG0AMAnE5eXp4iIyPP+Px1112nDz/8UH/961/10EMPqX///rr66qt1zTXXyM3t/D6bjouLq9KCdc2aNav02GKxqGnTpg6d/306e/bsUWxsbKUPDyTbUPyK50/WqFGjU9oIDQ09ZwDds2ePmjVrdsr370zXqYqIiAgNGDDglOPnszBhxXWbNm1a6XhYWJhCQ0NP+5o/fw8qzqv4HuzcuVOSbc2F0wkKCpJk+5DnxRdf1P3336+oqCj16NFDw4YN0+jRoxUdHX3O2s/miiuu0Lhx41RSUqKVK1fqhRdeUEFBwSnf/82bN+uxxx7T/PnzlZubW+m5852CcLKK72eLFi1Oea5Vq1aaNWuW8vPz5e/vX+W2AQA1jwAPAHAq+/fvV05OzimB7WS+vr765ZdftGDBAv3888+aOXOmvvrqK/Xr10+zZ88+r1XMqzJv/Xz9eaG9CuXl5bW2svqZrmP8acG7uuxc34OKURufffbZaYP4yR8sTJgwQcOHD9fUqVM1a9YsPf7445o4caLmz5+vTp06VbvGhg0b2j/UGDJkiCIiIjRu3DilpKTo6quvliRlZ2erT58+CgoK0jPPPKOkpCT5+PhozZo1evDBB51+u0EAgOMxhB4A4FQ+++wzSTrnVlpubm7q37+/Xn31VW3ZskXPP/+85s+fbx/+fKYwXV0VvbYVDMNQampqpQW/QkNDlZ2dfcpr/9x7XZXaGjdurIMHD54ypaBi+HTjxo3Pu61zXWfnzp2nhEJHX6eqKq6bmppa6fiRI0eqPay9YopBZGSkBgwYcMrtz6vIJyUl6f7779fs2bO1adMmlZSU6JVXXrE/74iftTvuuENJSUl67LHH7B80LFy4UEeOHNHkyZN1zz33aNiwYRowYMBpRx6cbw0V38/t27ef8ty2bdsUERFB7zsAODECPADAacyfP1/PPvusEhMTdeONN57xvKNHj55yrGPHjpJsW4hJsoeQ0wXq6vj0008rhehvv/1Whw4d0uDBg+3HkpKStHz5cpWUlNiPTZs27ZTt5qpS25AhQ1ReXq6333670vHXXntNFoul0vUvxJAhQ5Senl5ppfaysjK99dZbCggIUJ8+fRxynarq37+/PDw89O6771Y6/ufvR1UMHDhQQUFBeuGFF1RaWnrK8xVbzhUUFJyy0n1SUpICAwPtP2eS7d/zQn/OPDw8dP/992vr1q364YcfJP0xkuDk0RMlJSV65513Tnm9v7//eQ2pj4mJUceOHfXJJ59UqnnTpk2aPXu2hgwZckHvAwBQsxhCDwAwxYwZM7Rt2zaVlZUpIyND8+fP15w5c9S4cWP9+OOP8vHxOeNrn3nmGf3yyy8aOnSoGjdurMzMTL3zzjtq2LChevXqJckWtEJCQvTee+8pMDBQ/v7+6t69uxITE6tVb1hYmHr16qVbb71VGRkZev3119W0adNKW9399a9/1bfffqtBgwbp2muv1a5du/T555+fsjVYVWobPny4UlJS9Oijj2r37t3q0KGDZs+erR9++EETJkw4pe3quv322/X+++/rlltu0erVq5WQkKBvv/1WS5Ys0euvv37KHPzaEhUVpXvuuUevvPKKLr/8cg0aNEjr16/XjBkzFBERUa3e76CgIL377rsaNWqUOnfurJEjR6pBgwbau3evfv75Z/Xs2VNvv/22duzYof79++vaa69V69at5eHhoe+//14ZGRkaOXKkvb0uXbro3Xff1XPPPaemTZsqMjLyjPPrz+aWW27RE088oRdffFFXXnmlLr74YoWGhurmm2/W3XffLYvFos8+++y00yG6dOmir776Svfdd5+6du2qgIAADR8+/LTXeemllzR48GAlJydrzJgx9m3kgoOD9dRTT1W5bgBALTJxBXwAQD1UsdVXxc3Ly8uIjo42Lr30UuONN96otF1ZhT9vIzdv3jzjiiuuMGJjYw0vLy8jNjbWuP76640dO3ZUet0PP/xgtG7d2vDw8Ki0lVafPn2MNm3anLa+M20j99///td4+OGHjcjISMPX19cYOnSosWfPnlNe/8orrxhxcXGGt7e30bNnT2PVqlWntHm22v68jZxh2LY9u/fee43Y2FjD09PTaNasmfHSSy9V2irNMGzbk40dO/aUms60vd2fZWRkGLfeeqsRERFheHl5Ge3atTvtVndV3UbuTOdWfG/Pto2cYRhGWVmZ8fjjjxvR0dGGr6+v0a9fP2Pr1q1GeHi48be//e2U1/55i8KK6yxYsOCU4wMHDjSCg4MNHx8fIykpybjllluMVatWGYZhGFlZWcbYsWONli1bGv7+/kZwcLDRvXt34+uvv67UTnp6ujF06FAjMDDQkHTOLeXO9O9kGH9sj1dR65IlS4wePXoYvr6+RmxsrPGPf/zDmDVr1invJy8vz7jhhhuMkJAQQ5L9Z+h028gZhmHMnTvX6Nmzp+Hr62sEBQUZw4cPN7Zs2XLWugEA5rMYRj1a1QYAANQJ2dnZCg0N1XPPPadHH33U7HIAAKgVzIEHAABOrbCw8JRjr7/+uiSdsuAcAAB1GXPgAQCAU/vqq680efJkDRkyRAEBAVq8eLH++9//6rLLLlPPnj3NLg8AgFpDgAcAAE6tffv28vDw0L/+9S/l5ubaF7Z77rnnzC4NAIBaxRx4AAAAAABcAHPgAQAAAABwAQR4AAAAAABcAHPgJVmtVh08eFCBgYGyWCxmlwMAAAAAqOMMw9Dx48cVGxsrN7fz61snwEs6ePCg4uPjzS4DAAAAAFDP7Nu3Tw0bNjyvcwnwkgIDAyXZvnFBQUEmVwMAAAAAqOtyc3MVHx9vz6PngwAv2YfNBwUFEeABAAAAALWmKtO4WcQOAAAAAAAXQIAHAAAAAMAFEOABAAAAAHABzIEHAAAAgHMwDENlZWUqLy83uxS4CHd3d3l4eDh0q3ICPAAAAACcRUlJiQ4dOqSCggKzS4GL8fPzU0xMjLy8vBzSHgEeAAAAAM7AarUqLS1N7u7uio2NlZeXl0N7VFE3GYahkpISHT58WGlpaWrWrJnc3C58BjsBHgAAAADOoKSkRFarVfHx8fLz8zO7HLgQX19feXp6as+ePSopKZGPj88Ft8kidgAAAABwDo7oPUX94+ifG34KAQAAAABwAQR4AAAAAABcAAEeAAAAAFAlkydPVkhIyAW3Y7FYNHXq1Atup74gwAMAAABAPXTLLbfoyiuvNLsMVAEBHgAAAAAAF0CABwAAAABU8uqrr6pdu3by9/dXfHy87rrrLuXl5Z1y3tSpU9WsWTP5+Pho4MCB2rdvX6Xnf/jhB3Xu3Fk+Pj5q0qSJnn76aZWVldXW26hzCPAAAAAAgErc3Nz05ptvavPmzfrkk080f/58/eMf/6h0TkFBgZ5//nl9+umnWrJkibKzszVy5Ej787/++qtGjx6te+65R1u2bNH777+vyZMn6/nnn6/tt1NnEOABAAAAAJVMmDBBKSkpSkhIUL9+/fTcc8/p66+/rnROaWmp3n77bSUnJ6tLly765JNPtHTpUv3222+SpKeffloPPfSQbr75ZjVp0kSXXnqpnn32Wb3//vtmvKU6wcPsAgAAAAAAzmXu3LmaOHGitm3bptzcXJWVlamoqEgFBQXy8/OTJHl4eKhr167217Rs2VIhISHaunWrunXrpvXr12vJkiWVetzLy8tPaQfnjwAPAAAAALDbvXu3hg0bpjvvvFPPP/+8wsLCtHjxYo0ZM0YlJSXnHbzz8vL09NNP6+qrrz7lOR8fH0eXXS8Q4AEAAAAAdqtXr5bVatUrr7wiNzfbrOs/D5+XpLKyMq1atUrdunWTJG3fvl3Z2dlq1aqVJKlz587avn27mjZtWnvF13EEeAAAAACop3JycrRu3bpKxyIiIlRaWqq33npLw4cP15IlS/Tee++d8lpPT0+NHz9eb775pjw8PDRu3Dj16NHDHuifeOIJDRs2TI0aNdI111wjNzc3rV+/Xps2bdJzzz1XG2+vzmEROwAAAACopxYuXKhOnTpVun322Wd69dVX9eKLL6pt27b64osvNHHixFNe6+fnpwcffFA33HCDevbsqYCAAH311Vf25wcOHKhp06Zp9uzZ6tq1q3r06KHXXntNjRs3rs23WKdYDMMwzC7CbLm5uQoODlZOTo6CgoLMLgcAAACAkygqKlJaWpoSExOZt40qO9vPT3VyKD3wAAAAAAC4AAI8AAAAAAAugAAPAAAAAIALYBV6AABcWFpWvvKLyxzerr+3hxIj/B3eLgAAqD4CPAAALiotK18pLy+ssfYX/L0vIR4AACdCgAcAwEVV9LyPTWmquBBfh7V7ILtQkxak1kjPPgAAqD4CPAAALi4uxJeecgAA6gEWsQMAAAAAwAXQAw8AAAAAVXQgu1DH8ktq7Xqh/l4OnS4F10SABwAAAIAqOJBdqP6vLFRRqbXWrunj6aZ59/ettRC/e/duJSYmau3aterYseN5vWby5MmaMGGCsrOzTa2jLiPAAwAAAEAVHMsvUVGp1eGLiJ5JxeKix/JLqny9ffv26cknn9TMmTOVlZWlmJgYXXnllXriiScUHh5+xtfFx8fr0KFDioiIOO9rXXfddRoyZEiV6kPVEOABAAAAoBqcfRHR33//XcnJyWrevLn++9//KjExUZs3b9YDDzygGTNmaPny5QoLCzvldSUlJfLy8lJ0dHSVrufr6ytfX4b51yQWsQMAAACAOmjs2LHy8vLS7Nmz1adPHzVq1EiDBw/W3LlzdeDAAT366KOSpISEBD377LMaPXq0goKCdPvtt2v37t2yWCxat26dvb0ff/xRzZo1k4+Pj1JSUvTJJ5/IYrHYh8xPnjxZISEh9vOfeuopdezYUZ999pkSEhIUHByskSNH6vjx4/ZzZs6cqV69eikkJETh4eEaNmyYdu3aVRvfHpdEgAcAAACAOubo0aOaNWuW7rrrrlN6xaOjo3XjjTfqq6++kmEYkqSXX35ZHTp00Nq1a/X444+f0l5aWpquueYaXXnllVq/fr3uuOMO+wcAZ7Nr1y5NnTpV06ZN07Rp07Ro0SL985//tD+fn5+v++67T6tWrdK8efPk5uamq666SlZr7a0v4EoYQg8AAAAAdczOnTtlGIZatWp12udbtWqlY8eO6fDhw5Kkfv366f7777c/v3v37krnv//++2rRooVeeuklSVKLFi20adMmPf/882etw2q1avLkyQoMDJQkjRo1SvPmzbO/bsSIEZXO/89//qMGDRpoy5Ytatu27fm/4XqCHngAAAAAqKMqetjP5aKLLjrr89u3b1fXrl0rHevWrds5201ISLCHd0mKiYlRZmam/fHOnTt1/fXXq0mTJgoKClJCQoIkae/evedVd31DgAcAAACAOqZp06ayWCzaunXraZ/funWrQkND1aBBA0mSv3/NLMbn6elZ6bHFYqk0PH748OE6evSo/v3vf2vFihVasWKFJNtCejgVAR4AAAAA6pjw8HBdeumleuedd1RYWFjpufT0dH3xxRe67rrrZLFYzqu9Fi1aaNWqVZWOrVy58oJqPHLkiLZv367HHntM/fv3tw/rx5kxBx4AAAAAquFAduG5TzLxOm+//bYuvvhiDRw4UM8991ylbeTi4uLOOX/9ZHfccYdeffVVPfjggxozZozWrVunyZMnS9J5fwjwZ6GhoQoPD9cHH3ygmJgY7d27Vw899FC12qovCPAAAAAAUAWh/l7y8XTTpAWptXZNH083hfp7Vek1zZo106pVq/Tkk0/q2muv1dGjRxUdHa0rr7xSTz755Gn3gD+TxMREffvtt7r//vv1xhtvKDk5WY8++qjuvPNOeXt7V/XtSJLc3Nz05Zdf6u6771bbtm3VokULvfnmm+rbt2+12qsPLMb5rmpQh+Xm5io4OFg5OTkKCgoyuxwAAM7LpgM5GvbWYr1wVTslRjhu7mJaVr4e+X6jpo3vpbZxwQ5rFwBcUVFRkdLS0pSYmCgfHx/78QPZhTqWX3vztEP9vRQX4nvuE2vR888/r/fee0/79u0zuxSndaafH6l6OZQeeAAAAACoorgQX6cL1DXtnXfeUdeuXRUeHq4lS5bopZde0rhx48wuq14hwAMAAAAAzmnnzp167rnndPToUTVq1Ej333+/Hn74YbPLqlcI8AAAAACAc3rttdf02muvmV1GvcY2cgAAAAAAuABTA/y7776r9u3bKygoSEFBQUpOTtaMGTPszxcVFWns2LEKDw9XQECARowYoYyMjEpt7N27V0OHDpWfn58iIyP1wAMPqKysrLbfCgAAAAAANcrUAN+wYUP985//1OrVq7Vq1Sr169dPV1xxhTZv3ixJuvfee/XTTz/pm2++0aJFi3Tw4EFdffXV9teXl5dr6NChKikp0dKlS/XJJ59o8uTJeuKJJ8x6SwAAAAAA1AhT58APHz680uPnn39e7777rpYvX66GDRvqo48+0pQpU9SvXz9J0scff6xWrVpp+fLl6tGjh2bPnq0tW7Zo7ty5ioqKUseOHfXss8/qwQcf1FNPPSUvr9Pvk1hcXKzi4mL749zc3Jp7kwAAAAAAOIDTzIEvLy/Xl19+qfz8fCUnJ2v16tUqLS3VgAED7Oe0bNlSjRo10rJlyyRJy5YtU7t27RQVFWU/Z+DAgcrNzbX34p/OxIkTFRwcbL/Fx8fX3BsDAAAAAMABTF+FfuPGjUpOTlZRUZECAgL0/fffq3Xr1lq3bp28vLwUEhJS6fyoqCilp6dLktLT0yuF94rnK547k4cfflj33Xef/XFubi4hHgAAAMD5y94nFRypvev5hUshZJb6zvQA36JFC61bt045OTn69ttvdfPNN2vRokU1ek1vb295e3vX6DUAAAAA1FHZ+6RJXaXSwtq7pqevNHYlIf4CJCQkaMKECZowYYLD2uzbt686duyo119/3WFtno3pAd7Ly0tNmzaVJHXp0kUrV67UG2+8oeuuu04lJSXKzs6u1AufkZGh6OhoSVJ0dLR+++23Su1VrFJfcQ4AAAAAOFTBEVt4v+R+KbgWAnXOPunXV2zXPc8A/9577+mBBx7QsWPH5OFhi315eXkKDQ1Vz549tXDhQvu5CxcuVEpKilJTU5WUlHTGNivOO3bs2CkjpVE7TA/wf2a1WlVcXKwuXbrI09NT8+bN04gRIyRJ27dv1969e5WcnCxJSk5O1vPPP6/MzExFRkZKkubMmaOgoCC1bt3atPcAAAAAoB4IjpfCm5pdxWmlpKQoLy9Pq1atUo8ePSRJv/76q6Kjo7VixQoVFRXJx8dHkrRgwQI1atTorOHdkQzDUHl5uf2DhZpWUlJyxgXOXY2pi9g9/PDD+uWXX7R7925t3LhRDz/8sBYuXKgbb7xRwcHBGjNmjO677z4tWLBAq1ev1q233qrk5GT7D+Bll12m1q1ba9SoUVq/fr1mzZqlxx57TGPHjmWIPAAAAIB6q0WLFoqJiTmlp/2KK65QYmKili9fXul4SkqKPvvsM1100UUKDAxUdHS0brjhBmVmZkqSdu/erZSUFElSaGioLBaLbrnlFkm2TtiJEycqMTFRvr6+6tChg7799ttK7VssFs2YMUNdunSRt7e3Fi9erL59+2r8+PGaMGGCQkNDFRUVpX//+9/Kz8/XrbfeqsDAQDVt2lQzZsywt1VeXq4xY8bYr9WiRQu98cYbld77LbfcoiuvvFLPP/+8YmNj1aJFi9N+jz788EOFhIRo3rx5kqRNmzZp8ODBCggIUFRUlEaNGqWsrCz7+fn5+Ro9erQCAgIUExOjV155pRr/MhfG1ACfmZmp0aNHq0WLFurfv79WrlypWbNm6dJLL5Ukvfbaaxo2bJhGjBih3r17Kzo6Wt9995399e7u7po2bZrc3d2VnJysm266SaNHj9Yzzzxj1lsCAAAAAKeQkpKiBQsW2B8vWLBAffv2VZ8+fezHCwsLtWLFCqWkpKi0tFTPPvus1q9fr6lTp2r37t32kB4fH6///e9/kmwjow8dOmQPzhMnTtSnn36q9957T5s3b9a9996rm2666ZS1zR566CH985//1NatW9W+fXtJ0ieffKKIiAj99ttvGj9+vO6880793//9ny6++GKtWbNGl112mUaNGqWCggJJtg8LGjZsqG+++UZbtmzRE088oUceeURff/11pWvNmzdP27dv15w5czRt2rRTvjf/+te/9NBDD2n27Nnq37+/srOz1a9fP3Xq1EmrVq3SzJkzlZGRoWuvvdb+mgceeECLFi3SDz/8oNmzZ2vhwoVas2bNhfwTVZmpQ+g/+uijsz7v4+OjSZMmadKkSWc8p3Hjxpo+fbqjSwMAAAAAl5aSkqIJEyaorKxMhYWFWrt2rfr06aPS0lK99957kmxbcxcXFyslJUWNGjWyv7ZJkyZ688031bVrV+Xl5SkgIEBhYWGSpMjISPsc+OLiYr3wwguaO3eufapzkyZNtHjxYr3//vvq06ePvc1nnnnG3llboUOHDnrsscck2UZo//Of/1RERIRuu+02SdITTzyhd999Vxs2bFCPHj3k6empp59+2v76xMRELVu2TF9//XWlsO3v768PP/zwtEPnH3zwQX322WdatGiR2rRpI0l6++231alTJ73wwgv28/7zn/8oPj5eO3bsUGxsrD766CN9/vnn6t+/vyTbhw8NGzasyj/JBXO6OfAAAAAAgAvXt29f5efna+XKlTp27JiaN2+uBg0aqE+fPrr11ltVVFSkhQsXqkmTJmrUqJFWr16tp556SuvXr9exY8dktVolSXv37j3jGmOpqakqKCg4JZiXlJSoU6dOlY5ddNFFp7y+oideso2wDg8PV7t27ezHKrYJrxjKL0mTJk3Sf/7zH+3du1eFhYUqKSlRx44dK7Xbrl2704b3V155Rfn5+Vq1apWaNGliP75+/XotWLBAAQEBp7xm165d9ut0797dfjwsLOyMw/NrCgEeAAAAAOqgpk2bqmHDhlqwYIGOHTtm7w2PjY1VfHy8li5dqgULFqhfv37Kz8/XwIEDNXDgQH3xxRdq0KCB9u7dq4EDB6qkpOSM18jLy5Mk/fzzz4qLi6v03J/XJfP39z/l9Z6enpUeWyyWSscsFosk2T9M+PLLL/X3v/9dr7zyipKTkxUYGKiXXnpJK1asOOe1JOmSSy7Rzz//rK+//loPPfRQpfcxfPhwvfjii6e8JiYmRqmpqadtr7YR4AEAAACgjkpJSdHChQt17NgxPfDAA/bjvXv31owZM/Tbb7/pzjvv1LZt23TkyBH985//VHy8bau6VatWVWqroke7vLzcfqx169by9vbW3r17Kw2XrylLlizRxRdfrLvuust+bNeuXef9+m7dumncuHEaNGiQPDw89Pe//12S1LlzZ/3vf/9TQkLCaVfHT0pKkqenp1asWGGfanDs2DHt2LGjVt53BQI8AAAAAFRHzj6nv05KSorGjh2r0tLSSkGzT58+GjdunEpKSpSSkiIPDw95eXnprbfe0t/+9jdt2rRJzz77bKW2GjduLIvFomnTpmnIkCHy9fVVYGCg/v73v+vee++V1WpVr169lJOToyVLligoKEg333xztWs/nWbNmunTTz/VrFmzlJiYqM8++0wrV65UYmLiebdx8cUXa/r06Ro8eLA8PDw0YcIEjR07Vv/+9791/fXX6x//+IfCwsKUmpqqL7/8Uh9++KECAgI0ZswYPfDAAwoPD1dkZKQeffRRubnV7rrwBHgAAAAAqAq/cMnTV/q1FrcR8/S1XbeKUlJSVFhYqJYtW9rnk0u2AH/8+HH7dnOSNHnyZD3yyCN688031blzZ7388su6/PLL7a+Ji4vT008/rYceeki33nqrRo8ercmTJ+vZZ59VgwYNNHHiRP3+++8KCQlR586d9cgjj1z4+/6TO+64Q2vXrtV1110ni8Wi66+/XnfddVelrebOR69evfTzzz9ryJAhcnd31/jx47VkyRI9+OCDuuyyy1RcXKzGjRtr0KBB9pD+0ksv2YfaBwYG6v7771dOTo7D3+PZWAzDMGr1ik4oNzdXwcHBysnJUVBQkNnlAABwXjYdyNGwtxbrhavaKTHi9HP9qiMtK1+PfL9R08b3Utu4YIe1CwCuqKioSGlpaUpMTJSPj88fT2TvkwqO1F4hfuFSSHztXQ8OccafH1Uvh9IDDwAAAABVFRJPoEatq90B+wAAAAAAoFoI8AAAAAAAuAACPAAAAAAALoAADwAAAADnwNrfqA5H/9wQ4AEAAADgDDw9PSVJBQUFJlcCV1Txc1Pxc3ShWIUeAAAAAM7A3d1dISEhyszMlCT5+fnJYrGYXBWcnWEYKigoUGZmpkJCQuTu7u6QdgnwAAAAAHAW0dHRkmQP8cD5CgkJsf/8OAIBHgAAAADOwmKxKCYmRpGRkSotLTW7HLgIT09Ph/W8VyDAAwAAAMB5cHd3d3ggA6qCRewAAAAAAHABBHgAAAAAAFwAAR4AAAAAABdAgAcAAAAAwAUQ4AEAAAAAcAEEeAAAAAAAXAABHgAAAAAAF0CABwAAAADABRDgAQAAAABwAQR4AAAAAABcAAEeAAAAAAAXQIAHAAAAAMAFEOABAAAAAHABBHgAAAAAAFwAAR4AAAAAABdAgAcAAAAAwAUQ4AEAAAAAcAEEeAAAAAAAXAABHgAAAAAAF0CABwAAAADABRDgAQAAAABwAQR4AAAAAABcAAEeAAAAAAAXQIAHAAAAAMAFEOABAAAAAHABBHgAAAAAAFwAAR4AAAAAABdAgAcAAAAAwAUQ4AEAAAAAcAEEeAAAAAAAXAABHgAAAAAAF0CABwAAAADABRDgAQAAAABwAQR4AAAAAABcAAEeAAAAAAAXQIAHAAAAAMAFEOABAAAAAHABBHgAAAAAAFwAAR4AAAAAABdAgAcAAAAAwAUQ4AEAAAAAcAEEeAAAAAAAXAABHgAAAAAAF0CABwAAAADABRDgAQAAAABwAQR4AAAAAABcAAEeAAAAAAAXQIAHAAAAAMAFEOABAAAAAHABBHgAAAAAAFwAAR4AAAAAABdAgAcAAAAAwAUQ4AEAAAAAcAEEeAAAAAAAXAABHgAAAAAAF0CABwAAAADABRDgAQAAAABwAQR4AAAAAABcAAEeAAAAAAAXQIAHAAAAAMAFEOABAAAAAHABBHgAAAAAAFwAAR4AAAAAABdAgAcAAAAAwAUQ4AEAAAAAcAEEeAAAAAAAXICpAX7ixInq2rWrAgMDFRkZqSuvvFLbt2+vdE7fvn1lsVgq3f72t79VOmfv3r0aOnSo/Pz8FBkZqQceeEBlZWW1+VYAAAAAAKhRHmZefNGiRRo7dqy6du2qsrIyPfLII7rsssu0ZcsW+fv728+77bbb9Mwzz9gf+/n52e+Xl5dr6NChio6O1tKlS3Xo0CGNHj1anp6eeuGFF2r1/QAAAAAAUFNMDfAzZ86s9Hjy5MmKjIzU6tWr1bt3b/txPz8/RUdHn7aN2bNna8uWLZo7d66ioqLUsWNHPfvss3rwwQf11FNPycvLq0bfAwAAAAAAtcGp5sDn5ORIksLCwiod/+KLLxQREaG2bdvq4YcfVkFBgf25ZcuWqV27doqKirIfGzhwoHJzc7V58+bTXqe4uFi5ubmVbgAAAAAAODNTe+BPZrVaNWHCBPXs2VNt27a1H7/hhhvUuHFjxcbGasOGDXrwwQe1fft2fffdd5Kk9PT0SuFdkv1xenr6aa81ceJEPf300zX0TgAAAAAAcDynCfBjx47Vpk2btHjx4krHb7/9dvv9du3aKSYmRv3799euXbuUlJRUrWs9/PDDuu++++yPc3NzFR8fX73CAQAAAACoBU4xhH7cuHGaNm2aFixYoIYNG5713O7du0uSUlNTJUnR0dHKyMiodE7F4zPNm/f29lZQUFClGwAAAAAAzszUAG8YhsaNG6fvv/9e8+fPV2Ji4jlfs27dOklSTEyMJCk5OVkbN25UZmam/Zw5c+YoKChIrVu3rpG6AQAAAACobaYOoR87dqymTJmiH374QYGBgfY568HBwfL19dWuXbs0ZcoUDRkyROHh4dqwYYPuvfde9e7dW+3bt5ckXXbZZWrdurVGjRqlf/3rX0pPT9djjz2msWPHytvb28y3BwAAAACAw5jaA//uu+8qJydHffv2VUxMjP321VdfSZK8vLw0d+5cXXbZZWrZsqXuv/9+jRgxQj/99JO9DXd3d02bNk3u7u5KTk7WTTfdpNGjR1faNx4AAAAAAFdnag+8YRhnfT4+Pl6LFi06ZzuNGzfW9OnTHVUWAAAAAABOxykWsQMAAAAAAGdHgAcAAAAAwAUQ4AEAAAAAcAEEeAAAAAAAXAABHgAAAAAAF0CABwAAAADABRDgAQAAAABwAQR4AAAAAABcAAEeAAAAAAAXQIAHAAAAAMAFEOABAAAAAHABBHgAAAAAAFwAAR4AAAAAABdAgAcAAAAAwAUQ4AEAAAAAcAEEeAAAAAAAXAABHgAAAAAAF0CABwAAAADABRDgAQAAAABwAQR4AAAAAABcAAEeAAAAAAAXQIAHAAAAAMAFEOABAAAAAHABBHgAAAAAAFwAAR4AAAAAABdAgAcAAAAAwAUQ4AEAAAAAcAEEeAAAAAAAXAABHgAAAAAAF0CABwAAAADABRDgAQAAAABwAQR4AAAAAABcAAEeAAAAAAAXQIAHAAAAAMAFEOABAAAAAHABBHgAAAAAAFwAAR4AAAAAABdAgAcAAAAAwAUQ4AEAAAAAcAEEeAAAAAAAXAABHgAAAAAAF0CABwAAAADABRDgAQAAAABwAQR4AAAAAABcAAEeAAAAAAAXQIAHAAAAAMAFEOABAAAAAHABBHgAAAAAAFwAAR4AAAAAABdAgAcAAAAAwAUQ4AEAAAAAcAEEeAAAAAAAXAABHgAAAAAAF0CABwAAAADABRDgAQAAAABwAQR4AAAAAABcAAEeAAAAAAAXQIAHAAAAAMAFEOABAAAAAHABBHgAAAAAAFwAAR4AAAAAABfgYXYBAAAA9cqRXVLxcce26R0ohSc5tk0AgNMhwAMAANSWI7uktzrXTNvj1xDiAaCOI8ADAADUloqe90vul4LjHdNmzj7p11cc36sPAHA6BHgAAIDaFhwvhTc1uwoAgIthETsAAAAAAFwAAR4AAAAAABfAEHoAAEyy/1iBCkvK5evlLn8vD/l6ucvbw00Wi8Xs0gAAgBMiwAMAUItyCkr144aD+nb1fq3fl33K8/Fhvrr9kib6v4vi5ePpXvsFAgAAp0WABwCgFuw5kq9/zdquOZszVFJulSS5u1kU5OOhgpJyFZfZju07WqjHf9ist+an6vbeTXRD90by8+LXNQAAIMADAFDjZm5K1wPfrNfx4jJJUsvoQF3TpaGu6BinBoHekqSycqvyisv04/qDem/hLh3MKdJzP2/Vuwt36Z0bO6t7k3Az3wIAAHACBHgAAGpIablV/5q5Tf/+NU2S1DomSLf3bqKkBv6yWCzKyC1SRm5Rpdd0bhSqSTd21oJtmfp61X6l5xZp1Ee/6eEhLdU1IazSuamZebX2XgAAgPkI8AAA1ICM3CKNm7JGK3cfsx/bcihXE75aV+W2SsqtevqnLWd83seTTWUAAKgPCPAAADjYkbxijfxgudKy8hXo7aHx/ZrqhRnbNDalqeJCfKvUVrnV0Ler92n9/hxZJF3eMVbdE/8YTu/j6aaY4Kq1Wecc2SUVH3d8u96BUniS49sFAKCaCPAAADhQQUmZ/vLJKqVl5SsuxFef/7W78k/MfY8L8VVihH+V2/zHoJaavHS35mzJ0A/rDioiwFsDWkU5unTXdGSX9Fbnmmt//BpCPADAaRDgAQBwkLJyq8ZPWav1+7IV4uepT/7STYkR/tp0IOeC2nWzWHTrxQny9XTXj+sP6tNlu9UsMkCNw6v+YUCdU9Hzfsn9UnC849rN2Sf9+krN9OwDAFBNBHgAABzAMAw9/sMmzduWKW8PN304+iI1jQxwWPsWi0Uju8Zr/7ECrdmbrTfn79TzV7Zjr/gKwfFSeFOzqwAAoEax6g0AAA7w1vxU/fe3fXKzSG9e30kX/WnFeEewWCy6o0+SQvw8dTC7SJ8t3+PwawAAAOdFgAcA4AIt3ZWlV+fskCQ9fUVbDWwTXWPXCvLx1Ni+TWWRNH9bppb/fqTGrgUAAJwLAR4AgAuQW1SqB77ZIEm6vlu8RvVoXOPXbBsXrCs6xkqS/v3r7zp8vOgcrwAAAHUBc+ABAPiTtKx8+8rx5/LanB06kF2o6CAfXd2p4WkXrEvNzHN0iRrRpaE2H8zVzsw8vbtolx4f2loWi8Xh1wEAAM6DAA8AwEnSsvKV8vLCKr8uPbdI//f+srOe4+PpuIFvHm5uGpfSVA98u0FbDx3Xqj3H1LUG5t0DAADnQYAHAOAkFT3vY1OaKi7E94znHS8q1RvzdqqgpFy9mzXQoLZnn/fu4+mmmOAzt1cdkUE+GtIuRlPXHdCUFXvVKT5EHu7MjgMAoK4y9bf8xIkT1bVrVwUGBioyMlJXXnmltm/fXumcoqIijR07VuHh4QoICNCIESOUkZFR6Zy9e/dq6NCh8vPzU2RkpB544AGVlZ3f0EcAAE4nLsRXiRH+p70lhPtp1uYMFZSUq1GYn/56SeIZz624OTq8V7i8Q6yCfD2VnlukuVsza+QaAADAOZga4BctWqSxY8dq+fLlmjNnjkpLS3XZZZcpPz/ffs69996rn376Sd98840WLVqkgwcP6uqrr7Y/X15erqFDh6qkpERLly7VJ598osmTJ+uJJ54w4y0BAOqBFWlHtWbvMXm4WXRX3yR5mtjr7evlrv/r0lCS9L81+8977j4AAHA9pg6hnzlzZqXHkydPVmRkpFavXq3evXsrJydHH330kaZMmaJ+/fpJkj7++GO1atVKy5cvV48ePTR79mxt2bJFc+fOVVRUlDp27Khnn31WDz74oJ566il5eXmZ8dYAAHVUSZlVX6yw7b9+ecdYNQ73N7kiKaVFpGZuSteB7EJNXXdAN3av+ZXwAQBA7XOqiXI5ObaVe8PCbIvwrF69WqWlpRowYID9nJYtW6pRo0Zatsy2UNCyZcvUrl07RUVF2c8ZOHCgcnNztXnz5tNep7i4WLm5uZVuAACcj+kbDykrr0Rh/l66vEOs2eVIktzdLLqheyNJ0sxN6WwrBwBAHeU0Ad5qtWrChAnq2bOn2rZtK0lKT0+Xl5eXQkJCKp0bFRWl9PR0+zknh/eK5yueO52JEycqODjYfouPj3fwuwEA1EVH80s0dd0BSdL13RrJ28Pd5Ir+0Ck+RG1ig1RmNfTlyn1mlwMAAGqA0wT4sWPHatOmTfryyy9r/FoPP/ywcnJy7Ld9+/hDBwBwbl+v2qfiMquaRQaoZ1K42eVUYrFYdFOPxrJIWrrriPYdLTC7JAAA4GBOEeDHjRunadOmacGCBWrYsKH9eHR0tEpKSpSdnV3p/IyMDEVHR9vP+fOq9BWPK875M29vbwUFBVW6AQBwNrsO52nRjsOSpNHJjWWxWEyu6FQJ4f72veB/3njI5GoAAICjmRrgDcPQuHHj9P3332v+/PlKTEys9HyXLl3k6empefPm2Y9t375de/fuVXJysiQpOTlZGzduVGbmH1vnzJkzR0FBQWrdunXtvBEAQJ1mGIY+W2ZbuK5X0wg1jQw0uaIzG9Y+RpK0ODVLR/NLTK4GAAA4kqkBfuzYsfr88881ZcoUBQYGKj09Xenp6SosLJQkBQcHa8yYMbrvvvu0YMECrV69WrfeequSk5PVo0cPSdJll12m1q1ba9SoUVq/fr1mzZqlxx57TGPHjpW3t7eZbw8AUEesSDuq7RnH5e3hppFdnXvdlGZRgWoRFahyq6GZm+iFBwCgLjE1wL/77rvKyclR3759FRMTY7999dVX9nNee+01DRs2TCNGjFDv3r0VHR2t7777zv68u7u7pk2bJnd3dyUnJ+umm27S6NGj9cwzz5jxlgAAdYzVMPTt6v2SpKHtYxQe4PwfDg/rYOuFn7s1UwUl7AsPAEBdYeo+8IZhnPMcHx8fTZo0SZMmTTrjOY0bN9b06dMdWRoAAJKk39KO6kB2ofy83DWkbYzZ5ZyXzo1CFRvio4PZRZq/LVPD2jvHdncAAODCOMUidgAAOCOrYei7Nbbe98Fto+Xvbern3ufNzWLR0Ha20D5jU7rKrFaTKwIAAI5AgAcA4AxW7j6qfccK5evprkEu0vteoVfTCAX7eupofomW7TpidjkAAMABCPAAAJyGrff9gCRb73uAi/S+V/DycNOgNrbtVKdtOHRe09YAAIBzI8ADAHAa2w7lau/RAvl6umuwi/W+VxjQOkreHm7ae7RAmw7mml0OAAC4QAR4AABOY/62TEnSwDZRCvBxrd73CgHeHurTvIEkad7WDJOrAQAAF4oADwDAaRzMKZKPp5uGtHPN3vcK/VpGSpJW7Tmm3MJSk6sBAAAXggAPAMAZXNY6WoE+nmaXcUEah/srqYG/yq2Gftl52OxyAADABSDAAwBwkm3ptrni7m4WDW4bbXI1jpFyohd+wbZMFrMDAMCFEeABADjJD+sOSpI6NgxRiJ+XydU4xsVNIuTt4aaDOUXaln7c7HIAAEA1EeABADjhYHahlqRmSZIubhpucjWO4+vlrouTbO+nYnE+AADgegjwAACc8Mmy3bKeGGEeE+xrbjEOVrGY3Yq0I8orLjO5GgAAUB0EeAAAJBWUlOm/K/aaXUaNSWoQoPgwP5WWG1q8M8vscgAAQDUQ4AEAkPS/1fuVW1SmmGAfs0upERaLRf1a2Hrh529nMTsAAFwRAR4AUO9ZrYY+XrJbknR5h1hzi6lBvZpFyNPdon1HC7TrcL7Z5QAAgCoiwAMA6r1FOw7r96x8Bfp4aECrKLPLqTEB3h7qnmhbzG7hdhazAwDA1RDgAQD13keL0yRJI7vGy9fL3eRqalaf5g0kScvTjqis3GpyNQAAoCoI8ACAeu33w3lanJolN4s0OjnB7HJqXOuYIIX4eiq/uFzr9+eYXQ4AAKgCAjwAoF77auU+SVLfFpGKD/MzuZqa5+ZmUfKJPeEr9rwHAACugQAPAKi3Ssqs+nb1fkm24fP1Rc+mEZKk1XuOqbCk3ORqAADA+SLAAwDqrTlbMnQkv0SRgd7q1zLS7HJqTZMIf0UH+aik3KpVe46aXQ4AADhPBHgAQL315cq9kqRrL4qXh3v9+ZVosVjsvfAMowcAwHXUn79WAAA4yd4jBfp1py28XlePhs9X6NnUNg9+44Ec5RSWmlwNAAA4HwR4AEC99NUqW+/7Jc0i6sXidX8WE+yrpAb+shrS8t+PmF0OAAA4DwR4AEC9U1pu1derbIvXXd+tkcnVmOfiJIbRAwDgSgjwAIB6Z/62TB0+XqyIAC8NaBVldjmmSU4Kl8Ui7czMU0ZukdnlAACAcyDAAwDqnS9/sw2fH9Globw86u+vwlA/L7WJDZYkLd3FMHoAAJxd/f2rBQBQLx3MLtTCHYclSSO71t/h8xV6JtkWs2MYPQAAzo8ADwCoV75fe0CGIXVPDFNihL/Z5Ziua0KY3N0sOpBdqAPZhWaXAwAAzoIADwCoNwzD0PdrD0iSRnRuaHI1zsHf20Pt4mzD6H9LO2pyNQAA4GwI8ACAemPTgVylZubJ28NNg9pFm12O0+iWGCZJWpHGPHgAAJwZAR4AUG9U9L4PaB2lIB9Pk6txHhc1DpWbRdpzpIDV6AEAcGIEeABAvVBWbtWP6w9Kkq7uFGdyNc4l0MdTrU+sRr+CYfQAADgtAjwAoF74NTVLWXnFCvP3Uu/mDcwux+l0S7ANo/+NYfQAADgtAjwAoF74fo1t+Pzw9jHydOfX3591TQiVRdKuw/k6VlBidjkAAOA0+AsGAFDn5RWXafaWdEnSVaw+f1ohfl5qGRMoSdp8MNfkagAAwOkQ4AEAdd7MTekqKrWqSYS/OjQMNrscp9UtIVyStPlAjsmVAACA0yHAAwDqvO/X7pckXdUpThaLxeRqnFfFdnJ7jhaYXAkAADgdAjwAoE47lFOopbtsC7NdyerzZxXm76VmkQFmlwEAAM6AAA8AqNN+XHdQhmFbpC0+zM/scpxe98Rws0sAAABnQIAHANRpP288JEm6vCO97+ejYhi9JOUUlppYCQAA+DMCPACgztp7pEAb9ufIzSINbhttdjkuoUGgt2JDfCRJv6UdNbkaAABwMgI8AKDOquh9T04KV0SAt8nVuI5WMUGSpBW/HzG5EgAAcLJqBfgmTZroyJFTf6lnZ2erSZMmF1wUAACOMP1EgB/SLsbkSlxL6xMBfu2+bBWWlJtcTR1lLZdyD0rHdktZO6SMTVL6Bik/SzIMs6sDADgpj+q8aPfu3SovP/UXenFxsQ4cOHDBRQEAcKH2HMnXxgO24fOD2jB8viqig2xD6IvLrPp152FdxvfvwhmGlLVT2vSt7fGsR6Ty4tOf6x0khSVKoYlSbCcppqPk5l5rpQIAnFeVAvyPP/5ovz9r1iwFBwfbH5eXl2vevHlKSEhwWHEAAFRXxfD5i5MiFM7w+SqxWCz2+3O2ZBDgL4RhSFt+kBZOlA5v++N4ebHk4S15+ErunpK7l2RYpbwMqThXOrTedtsyVfINk5r0lZL6SaEJJr0RAIAzqFKAv/LKKyXZfrHffPPNlZ7z9PRUQkKCXnnlFYcVBwBAdf28wRbgh7Zn+PyFmLctU+VWQ+5ulnOfjD8YhrRrvjTvGenQOtsxd28puq10YLV0yd+lxN6S5U+zGcuKpZx90tHfbUPr9yyVCo9Km7+z3SJaSB1vkGI7Sxb+TQCgvqlSgLdarZKkxMRErVy5UhERETVSFAAAFyItK1+bD+bK3c2igfQeV1uAt4eO5pdo9Z5jlbaXwzlk75V+GCul/WJ77BUgJY+Tku+SjqZJH/SRghueGt4lW698eFPbrdllUrc7bIF/1zxp/0opa7s090kpso3UeZQU1bZ23xsAwFTVmgOflpbm6DoAAHCY6fbh8+EK8/cyuRrX1TUhVAu2H9acLekE+PP1+0Lpm1ttvebu3lLXv0qX3Cf5V7PTw91TatTDdivMljZ9I22bLmVulmY+JMV1kZoNdOQ7AAA4sWoFeEmaN2+e5s2bp8zMTHvPfIX//Oc/F1wYAADVNe3E8PlhDJ+/IN2bhGvB9sOavSVDjwxpVWluPP7EMKSlb0pzn7LNZY/pKF37iWPnrPuGSF1vk1pfJW34Sto529Y7f2i97fnyUsddCwDglKq1jdzTTz+tyy67TPPmzVNWVpaOHTtW6QYAgFl+P5ynrYdy5eFm0WWtGT5/ITo3CpGXh5v2HCnQzsw8s8txXiX50re3SnOesIX3jjdKf5lZcwvO+UdIyWOlK9+zzYW3ltmOf3ebtO+3mrkmAMApVKsH/r333tPkyZM1atQoR9cDAMAFqRg+37NphEIZPn9B/Lw81DMp/MQw+gw1jwo0uyTnU1ooTblO2v2r5OYpDf6ndNGY2llgLihGGvC0tPFrae1n0rE06aPLbOG+/xO2+fQAgDqlWj3wJSUluvjiix1dCwAAF2z6xnRJ0tB2DJ93hEtPjGKYvSXD5EqcUFmJ9PVoW3j3CpRu/sk25702pxpYLLZ58JLUfJAkQ1r2tvTv/lLmtrO+FADgeqoV4P/6179qypQpjq4FAIALsu9ogbYcypWbRRrQOsrscuqEAa0iJUnr92UrI7fI5GqciLXcNmR952zbXu43fi01Tja3pr4PS9d/KfmFSxkbbavd//Zv2/x8AECdUK0h9EVFRfrggw80d+5ctW/fXp6enpWef/XVVx1SHAAAVVHRS9wtMYzV5x0kMshHnRqFaO3ebM3dmqEbuzc2uyTzWa3Sj3dLW6bahs2P/Fxq7CQjE1sMlu5cJk2907b13PS/2/ajv/Jd2yJ4AACXVq0e+A0bNqhjx45yc3PTpk2btHbtWvtt3bp1Di4RAIDzM2uzbfg8i9c51oBWttEM87dmmlyJk5j7pLTuc8niLl3zH6npALMrqiwwSrrxW2nQi5K7l7R9uvRBX+nQBrMrAwBcoGr1wC9YsMDRdQAAcEGO5BVr1e6jkqTL2jB83pH6t4rUS7O2a3FqlgpLyuXr5W52SebZ8oNtuzhJumKS1Ppyc+s5Ezc3qcffpPhu0tc3n1jg7lJp6KtSpxvNrg4AUE3V6oEHAMDZzNuaKashtYkNUsNQP7PLqVNaRAUqLsRXxWVWLUnNMrsc8xzZJU0da7vf8x6p4/Xm1nM+4jpLdyySml4qlRVJP9xlG/5fynoGAOCKqtUDn5KSIstZVlidP39+tQsCAKA6Zm+xDZ8f2Ibh845msVjUv1WkPl22R/O2ZdTPBQJLC20rzpcclxr3lPo9YXZF588vTLrha+nXl6UFL0hrPpEOrZeu/VQKZU0DAHAl1eqB79ixozp06GC/tW7dWiUlJVqzZo3atWvn6BoBADir/OIy/bLT1jPM8Pma0f/EPPh5WzNl1MdVzaf/XcrYJPk3sM17d69WH4h53NykPv+Qbvqf5BsmHVonvd9b2jnH7MoAAFVQrd8+r7322mmPP/XUU8rLy7ugggAAOF9pWfnKLy7TktQslZRZFRPso9IyqzYdyKl2m6mZ/B47nR5NwuTn5a7M48XadCBX7RoGm11S7Vn7ue1mcZNGfCQFuvAoj6b9pTt+kb65WTqwWvri/2zBvs9DtpAPAHBqDv34+KabblK3bt308ssvO7JZAABOkZaVr5SXF1Y6diinSMPfXuKQ9n08CTMn8/Zw1yXNIjRrc4bmbs2oPwE+94D0899t91MekZr0MbceRwiJl26dIc16RFr5obToRdsK9Vd/IPkEmV0dAOAsHBrgly1bJh8fH0c2CQDAaeUXl0mS/tYnSZOXpqmo1KrbL2mihAj/C27bx9NNMcG+F9xOXdO/VZRmbc7QvG0ZuvfS5maXUzt+eVkqK5QSLpF63W92NY7j4S0NfUVq2FX66R5pxwzpwwHS9f+VwpPMrg4AcAbVCvBXX311pceGYejQoUNatWqVHn/8cYcUBgDA+Sgps6qo1KogX0/1ad5Abm5nXmQVF6Zfy0hZLNKmA7lKzylSdHA9+ND+4BrJw1e6/K26OcS8w0gporn05Y1S1nbp3ynOubc9AEBSNRexCw4OrnQLCwtT3759NX36dD355JOOrhEAgDPafNA2371Lo1DCew2LCPBWx/gQSdL8bZnmFlPTik5aR6Hfo1JYonm11LS4ztLtC6X47rb3/cX/SUvelOrjYoUA4OSq1QP/8ccfO7oOAACqZeuhXElS14RQkyupH/q3jNTavdmatzVDN3RvZHY5NcMwpI3f2u43aCl1v9PcempDYJR080/Sz/dLaz+T5jwupW+ULn9T8mQ6CQA4iwsaC7Z69Wp9/vnn+vzzz7V27VpH1QQAwHnLLSqTt4eb2sTWk0XVTFaxndzi1CwVlpSbXE0N2bNEythou9/7H663ZVx1eXjbpgoMfkmyuEsbv5b+M0jKOWB2ZQCAE6oV4DMzM9WvXz917dpVd999t+6++2516dJF/fv31+HDhx1dIwAAZ9W+YbC8POrg/GQn1DI6UHEhvious2rpriyzy3G84jxpxXt/PK5vC7pZLFL326XRU//YL/6DvtLeFSYXBgCQqhngx48fr+PHj2vz5s06evSojh49qk2bNik3N1d33323o2sEAOCsOjVi+HxtsVgs6t8qUpI0d2sdnAe/fopUlC0FRJpdibkSe0u3L5Ci2kr5mdInw/6YVgAAME21AvzMmTP1zjvvqFWrVvZjrVu31qRJkzRjxgyHFQcAwJkczS+x3+90YmE11I5+LW3hdv62DBl1aaGznH3Stp9t99uMMLcWZxCaIP1lltRymFReIv1vjLToJRa3AwATVSvAW61WeXp6nnLc09NTVqv1gosCAOBcVu4+KklqGOqrED8vk6upX3o0CZefl7sycou16UCu2eU4zqr/SEa51LCb1KCF2dU4B+8A6dpPpeRxtscLnpOm3iWVlZz9dQCAGlGtAN+vXz/dc889OnjwoP3YgQMHdO+996p///4OKw4AgDNZmWYL8C2jA02upP7x8XTXJc0iJElzt2aYXI2DHFgj7V9pW7ztojFmV+Nc3Nylgc9LQ1+1fX/WT5E+v7ryVnsAgFpRrQD/9ttvKzc3VwkJCUpKSlJSUpISExOVm5urt956y9E1AgBQSVFpudbuy5YktYwOMreYeqpiNfo6sR+8tVxa9aHtfsthUnCcufU4q65jpBu/kbyDpN2/Sh8PlY6nm10VANQr1doXJT4+XmvWrNHcuXO1bds2SVKrVq00YMAAhxYHAMDpLPv9iIrLbFO2YoJ9TK6mfkppESmLRdp4IEcZuUWKCnLhf4cds6TsvZJ3oNTherOrcW5N+0u3/Cx9PsK21d5Hl0mjvq9/q/UDgEmq1AM/f/58tW7dWrm5ubJYLLr00ks1fvx4jR8/Xl27dlWbNm3066+/1lStAABIkuaftPq5xWIxsZL6q0Ggtzo0DJEkzXPl1ehL8qR1n9vud7zRNucbZxfTXhozWwpNlLL3SP8ZKB1cZ3ZVAFAvVCnAv/7667rtttsUFHTqcMXg4GDdcccdevXVVx1WHAAAf2YYhubVlXnXLm7Aie3kXPrfY+M3UnGuFNxIaj7Y7GpcR1iiLcRHt5fyD0uTh0l7lpldFQDUeVUK8OvXr9egQYPO+Pxll12m1atXX3BRAACcybb04zqYUyRvj2ot4wIHqpgHvzg1S4Ul5SZXUw0FR6Wt02z3u9xiW6wN5y8g0jacPuESqeS4bVg9IR4AalSV/vrJyMg47fZxFTw8PHT48OELLgoAgDOp6O3twN7vpmsZHajYYB8Vl1m1dFeW2eVU3cavpfJi25ZxDbuaXY1r8gmyLWzXpK9Umn8ixC81uyoAqLOqFODj4uK0adOmMz6/YcMGxcTEXHBRAACcybwTq553SwgzuRJYLBZ7L/xcV5sHn5cp7Zhpu99ptMRaCtXn6Std/6XUJOVEiL9G2r3Y7KoAoE6q0ir0Q4YM0eOPP65BgwbJx6fyarOFhYV68sknNWzYMIcWCABAhay8Yq07sX3cRQmh5hYDSVL/VpH6bPkezd+WIcNo6zqLCm74UrKW2eZwx3Q483lZOxx7XUe35yw8faXr/yt9eYO0a770xf9JN/1Panyx2ZUBQJ1SpQD/2GOP6bvvvlPz5s01btw4tWjRQpK0bds2TZo0SeXl5Xr00UdrpFAAABZtPyzDkNrEBikiwNvsciCpR5Nw+Xm5KyO3WJsP5qptXLDZJZ1b7gEpda7tfqdRpz/H09f29bvbaqaGivbrEk9faeQU6csbpV3zpCkjpb/MkKLamF0ZANQZVQrwUVFRWrp0qe688049/PDDMgxDkm0I3cCBAzVp0iRFRUXVSKEAACzYbhumndIi0uRKUMHH0129mkZo9pYMzd2a4RoBft0UybDa5r1Htjr9OUFx0lXvS6WFjr++p6+t/brI01ca+YX02VXS3mXSZ1ef2HKusdmVAUCdUKUAL0mNGzfW9OnTdezYMaWmpsowDDVr1kyhoQxlBADUnLJyq37daVsorW+LBiZXg5MNaBWl2VsyNG9rpiYMaG52OWd3bLeU9ovtfsebzn5uXQ3ZNa1iOP1/BkuHt0qfXy39ZbbkH252ZQDg8qq9B09oaKi6du2qbt26Ed4BADVu/f5s5RSWKtjXUx1Zgd6ppLSMlMUibTyQo4zcIrPLObv1UyQZUuOeUniS2dXUXb6h0qjvpOB46UiqNOX/pOI8s6sCAJdn6ia6v/zyi4YPH67Y2FhZLBZNnTq10vO33HKLLBZLpduf96E/evSobrzxRgUFBSkkJERjxoxRXh6/IACgrlmwzbZN6SXNIuThzh7wzqRBoLc6NAyRJM3f5sSr0R/bc2KLM4vU4Qazq6n7gmKlm76zhfkDq6Vv/yJZy82uCgBcmql/AeXn56tDhw6aNGnSGc8ZNGiQDh06ZL/997//rfT8jTfeqM2bN2vOnDmaNm2afvnlF91+++01XToAoJYt3MH8d2fWv6Xt32Xe1gyTKzmLjV/bvjZOZk52bWnQXLrxW8nDR9o5S5r/nNkVAYBLq/IceEcaPHiwBg8efNZzvL29FR0dfdrntm7dqpkzZ2rlypW66KKLJElvvfWWhgwZopdfflmxsbEOrxkAUPsyjxdp04FcSVLv5sx/d0b9W0XplTk7tDg1S0Wl5fLxdDe7pMpyD0i7f7Xdb3edubXUNw0vki5/W/rur9LiV22r0re7xuyqAMAlOf0YxIULFyoyMlItWrTQnXfeqSNHjtifW7ZsmUJCQuzhXZIGDBggNzc3rVix4oxtFhcXKzc3t9INAOC8Fm23DZ9v3zBYDQLZPs4ZtYoJVGywj4pKrVqSmmV2Oafa+O0fK88z9732tf8/qec9tvs/jJMOrjO1HABwVU4d4AcNGqRPP/1U8+bN04svvqhFixZp8ODBKi+3zZ9KT09XZGTloZQeHh4KCwtTenr6GdudOHGigoOD7bf4+PgafR8AgAuzcIctwPel991pWSwW9W9l20p2nrPNg8/LlHbNt91vd625tdRn/Z+Umg6Qygpte8XnHTa7IgBwOU4d4EeOHKnLL79c7dq105VXXqlp06Zp5cqVWrhw4QW1+/DDDysnJ8d+27dvn2MKBgA4XFm5Vb+eCPB9mP/u1Pq1sv37zN+aKcMwTK7mJJv+JxnlUnT7M+/7jprn5i6N+EgKS5Jy90tfj5bKy8yuCgBcilMH+D9r0qSJIiIilJqaKkmKjo5WZmblT/nLysp09OjRM86bl2zz6oOCgirdAADOae2+bOUWlSnEj+3jnF1yk3D5ebkrPbdImw86yfS0gqPSztm2++1HmlsLJN8Q6fovJa9Aae9SadE/za4IAFyKSwX4/fv368iRI4qJiZEkJScnKzs7W6tXr7afM3/+fFmtVnXv3t2sMgEADrTgxHDs3s0ayN3NYnI1OBsfT3f1ahohSZrrLKvRb/5espZKDVpJ0e3MrgaSbWX6y9+w3f/lZen3RebWAwAuxNQAn5eXp3Xr1mndunWSpLS0NK1bt0579+5VXl6eHnjgAS1fvly7d+/WvHnzdMUVV6hp06YaOHCgJKlVq1YaNGiQbrvtNv32229asmSJxo0bp5EjR7ICPQDUEQtPLGCX0pL5765gwIl58E6xH3zxcWnHDNv99tdKFj4AchptR0idR0sypO9ul/KdcOFDAHBCpgb4VatWqVOnTurUqZMk6b777lOnTp30xBNPyN3dXRs2bNDll1+u5s2ba8yYMerSpYt+/fVXeXv/sQLxF198oZYtW6p///4aMmSIevXqpQ8++MCstwQAcKCM3CJtOZQri8XWAw/n1/fEBy0b9ucoI7fI3GK2/SyVFUmhiVLcRec+H7Vr0ItSRAspL12aeqdktZpdEQA4PVP3ge/bt+9ZF7mZNWvWOdsICwvTlClTHFkWAMBJ/LF9XIjCA9g+zhVEBvqoQ3yI1u/L1vxtmbq+WyNzCikrlrb9ZLvfdgS9787Iy0/6v4+lD1Js6xSseFdKHmt2VQDg1FxqDjwAoH5ZVLH6PNvHuZQBLW2r0c8zcx78rnlSUY7kHyklXGJeHTi7qDbSoIm2+3OelA5tMLceAHByBHgAgFMqtxpanGqbF0uAdy0V+8EvTs1SUWl57RdgLbctXidJba6ybV8G53XRX6SWw2yLDU69UyorMbsiAHBaBHgAgFNavz9bOYWlCvLxUIeGwWaXgypoFROo2GAfFZVatXSXCYuT7V0mHT8keQdKTS+t/eujaiwWadjrkl+4lLFJ+uUlsysCAKdFgAcAOKWK+e+XNGsgD3d+XbkSi8Wifq1sw+jnbq3l1egNQ9r0P9v9lsMkT5/avT6qJ6CBNPQV2/1fX5EOrjO1HABwVvxFBABwSr/stAX43s0jTK4E1VExjH7+1syzLljrcBkbpSM7JXcvW4CH62hzle1mlJ8YSl9sdkUA4HRMXYUeAIDTyS4o0fp92ZKk3sx/d0nJTcLl6+mu9NwibT6Yq7ZxtTQNoqL3vemlkg9TL1zOkFek3YulzC3Sohel/k+YXZF0ZJdUfNzx7XoHSuFJjm8XQJ1GgAcAOJ3FqVmyGlLzqADFBPuaXQ6qwcfTXb2aRWjOlgzN25pZOwH+2G7pwGrJ4ia1vrLmrwfH8w+Xhr0mfXWTtPh1qeVQKa6LefUc2SW91bnm2h+/hhAPoEoI8AAAp/PLie3jejej992VDWgVaQvw2zJ0z4BmNX/BLT/YvjZKloJiav56qBmthkttr5E2fSv9eI90+0LJ/Tz+ZK2JnvKsHbavl9wvBcc7rt2cfba5/jXRsw+gTiPAAwCcimEY+mWHbeVyhs+7tpQT+8Fv2J+jjNwiRQXV4IJyxcel3xfa7tP77voGvyjtmmdb02D5O1LPu89+fk33lEc0l4Liaq59ADhPBHgAgFPZkZGn9NwieXu4qVtimNnl4AJEBvqoQ3yI1u/L1oJtmRrZrVHNXWzPEts+4hEtpMhWNXcd1A7/COmy56QfxkoLJ0qtr5BCG5/5/IqebEf3lEuSpy/hHYDTIMADAJxKxfD5Hk3C5ePpbnI1uFD9W0Zq/b5szd1awwF+92Lb19ZX1Nw1ULs63iit+6+0Z7E0/e/SDV/b9ow/m+B4Kbxp7dQHACZgGzkAgFP5Y/s4hs/XBf1P7Ae/OPWwikrLa+5CJXmSfwOpcc+auwZql8ViW9DO3UvaOVvaMtXsigDAdAR4AIDTKCwp14q0o5KkPuz/Xie0jglSbLCPikqtWrory/EXOHmP+ZZDJTdGbdQpDZpLve6z3Z/xoFSUY249AGAyAjwAwGksTzuikjKr4kJ8ldQgwOxy4AAWi0X9TvTCz9ua6fgLHFxj++ruJTUb5Pj2Yb5e99qGxedlSPOeMbsaADAVAR4A4DTs28c1j5DlXHNd4TL6t4ySJM3flinj5B5zR9j4je1rw26SNx/61EmePrah9JK08iPp0Hpz6wEAExHgAQBOYxH7v9dJyUnh8vV016GcIm0+mOu4hrN2SnuX2e4n9nFcu3A+ib2ltiMkGdL0BypPnQCAeoQADwBwCvuOFuj3w/lyd7Po4qbMf69LfDzd1auZ7d/UocPol7/7x/0APvSp8y59VvL0k/atkDZ8ZXY1AGAKAjwAwClUrD7fKT5Ewb6eJlcDRxtQMQ9+W4ZjGiw4Kq3/r2PagmsIjpN6P2C7P+cJqciBozkAwEWwDzwAwCn8Mf+dntS6KKWFLcBv2J+jzNwiRQb5XFiDqydLpQW2xc2OpF54gTi9I7uk4uOOb9c7UApPqvrrksdKaz+Xju6SFr0oDXze8bUBgBMjwAMATFdabtXS1COSpD4E+DopMshHHRoGa/3+HM3flqmR3RpVv7HyUum3f9vut/s/aeFExxSJyo7skt7qXHPtj19T9RDv4S0NflH64hppxXtS59FSgxY1Ux8AOCECPADAdGv3Zut4cZlC/TzVNi7Y7HJQQ/q3itL6/Tmau/UCA/yWH6TjByX/SCmpHwG+QtaOmmnvkvul4HjHtZuzT/r1ler37De7VGoxRNo+XZrxD2nUVMfVBgBOjgAPADBdxfD5Xs0ayN2N7ePqqn4tI/XqnB1anHpYRaXl8vF0r3ojhiEtm2S73+022/7v9Z2nr+3rd7fVTPsRzaWguJppu7oGviClzpN+Xyht/VEKaWx2RQBQKwjwAADTVSxgx/D5uq1NbJBign10KKdIy3YdUUrLyKo3sm+FdHCN5O4tXfQXKWe/4wt1NUFx0lXvS6WFjm/b09f5wrskhSVKPe+RfvmXNOtRacSHZlcEALWCAA8AMNWRvGJtPJAjSerdjO3j6jKLxaJ+LSP1xYq9mrs1o3oBvqL3vcN1kn8EAb6CM4bsmtbrXttOBDn7pHVfmF0NANQKtpEDAJhqcWqWDENqGR144SuTw+kNaBUlSZq/LVOGYVTtxcd2S9um2e73uMuxhcH1ePn9sQr9+i/NrQUAagkBHgBgqkUn5r/3acHw+fogOSlcPp5uOpRTpM0Hq7iP94oPJMMqNUmRIlvVTIFwLa0ul5r0lcpLzK4EAGoFAR4AYBqr1dAvO7IkSX2aEeDrAx9Pd/Vqavu3nr8t8/xfWJQrrfnUdj95bA1UBpdksUiD/yVZTiyImLnF3HoAoIYR4AEAptmanqusvGL5erqrS0Ko2eWglgxoZZv7Pm9rxvm/aO3nUslx24roSf1rqDK4pAYtpHbX2O5v/l4qLzW3HgCoQQR4AIBpKnrfL04Kl7dHNbYUg0vqd2LxuvX7c5SZW3TuF1jLpRXv2e73uFNy488X/Ennm21f8w9LW6aaWgoA1CRWoQcAmGbRDtsQ6t5sH1evRAb5qH3DYG3Yn6P52zI1slujs79g289S9h7JN1RqP7J2ikTNy9rhuLZyD/xxf8NXtnUS/NnVAkDdQ4AHAJgir7hMq/cck0SAr48GtIrShv05mrMl49wBfvk7tq8X/cW28jhcm6ev7et3tzm+7bAk6eguafXHUu8HHN8+AJiMAA8AMMWyXUdUWm6oUZifEsIJZfXNwDbRenXODv26M0t5xWUK8D7DnyQH1kh7l0lunlLXGgh8qH1BcdJV70ulhY5t19PX1ua0e6W0RVLzQVJ0O8deAwBMRoAHAJiiYvh8n+YNZLFYTK4Gta15VIASwv20+0iBFm7P1LD2sac/saL3ve3VUlBM7RWImhUUV3NtNx8k7Zgh/fa+NOwNyY31NQDUHawCAwCodYZhaOH2E/u/M3y+XrJYLBrYNlqSNHNT+ulPyj1oW1VcknrcVUuVweV1GiV5B0rHdkvbp5tdDQA4FAEeAFDr0rLytf9Yobzc3ZScFG52OTDJoDa2AL9gW6aKSstPPeG3f0vWMqlxTym2Y+0WB9flEyR1HGW7v+5zqSjH3HoAwIEI8ACAWrdoh633vWtiqPzPNPcZdV6HhiGKCvJWfkm5lu7KqvxkSYFtITKJ3ndUXfOBUlgTqSRfWvOJ2dUAgMMQ4AEAta4iwDN8vn5zc7NoYJszDKNf/1+p8JgUmiC1GFz7xcG1ublL3f9mu79zjmO3rAMAExHgAQC1qqi0XMt/PyJJ6tM80uRqYLaKAD93a6bKyq22g1artPxd2/3ud7IIGaonsrVtP3gZ0or3JMNqdkUAcMEI8ACAWvVb2lEVlVoVHeSj5lEBZpcDk3VLDFOIn6eO5pdo5e5jtoOpc6UjOyXvIKnTjeYWCNfW5Vbb9nJZO6TUeWZXAwAXjAAPAKhVJw+fZ/s4eLq7aUCrKEnSrM0nhtEvn2T72nm0bTVxoLr8wqT219vur5ksleSZWg4AXCgCPACgVtkDfAvmv8OmYhj9rM3pMtI3Sb8vlCxuUrfbzS0MdUOr4VJwQ9tq9OummF0NAFwQAjwAoNbsP1ag1Mw8ubtZ1LNphNnlwElc0ixCfl7uOpRTpA3zTgSsVsOl0MbmFoa6wd1T6naH7f62abb94QHARRHgAQC15pcdtq3COsWHKNjX0+Rq4Cx8PN3V98SIjFnbTsyD7zHWxIpQ58R2khol2xayW/GeZBhmVwQA1UKABwDUmkU7MiWxfRxOZd9OrqyzjNguUnw3kytCndP1NsndW8rYJP0+3+xqAKBaCPAAgFpRWm7VktQT28cx/x1/kpIULE+V6XcjVqmtx0oscAhHC4iUOpxY0G7lR1LxcXPrAYBqIMADAGrFmj3HlFdcpjB/L7WNDTa7HDiZoNSp6um2SZI0q6itydWgzmpzpRTSSCrOlVZPNrsaAKgyD7MLAADUfWlZ+fp29X5JUvuGwdpyKPeC20zNZDuoOsMwpOXvapBbhBZaO2rm1kyNG9DC7KpQF7l5SD3ukmY+JO2cJTUdIEW2MrsqADhvBHgAQI1Ky8pXyssL7Y8Xbj+shdsPO6x9H08Gk7m8tF+kjE0a4B0lt3Jp04Fc7T9WoIahfmZXhrooqq3U9FIpdY60fJI07HVbsAcAF8D/rQAANSq/uKzS40eGtFKAt2N+/fh4uikm2NchbcFEy9+RJEV0vlwX7Q/Tb2lHNWtzhsb0SjS5MNRZXW6R9q2wbSm39UepzdVmVwQA54UADwCoNYkR/moXx/x3nCQrVdoxU5JF6nGnBm51swX4TekEeNQcn2Cpy63S0jekdV9IjXtKAVFmVwUA58S4QwBArenQMMTsEuBsVrxr+9p8kBSepIFtbCFq5Z6jOny82MTCUOc1HWAbTl9WLC2bxN7wAFwCAR4AUKPKrX/8Udwhnt53nKTgqLRuiu1+jzslSQ1D/dQuLliGIc3dmmFicajzLBYpeZzk5ikdXCP9vtDsigDgnAjwAIAaVbFavI+nm5pFBppcDZzKmk+k0gJbL2hib/vhil74WZvTzaoM9UVww5P2hv+3VJRjbj0AcA4EeABAjVqz95gkKalBgNzdLCZXA6dRXiqt+MB2v8ddtt7QEwa1jZYkLUnNUm5RqRnVoT5pe7UUmmDbG37lv82uBgDOigAPAKhRq/fYAnzzKHrfcZItP0jHD0r+kVK7ayo91TQyUE0a+Ku03NCCbZkmFYh6w81DSh4vWdxsw+j3rzK7IgA4IwI8AKDGZBeUaEfGcUlSs8gAk6uB0zAMaelbtvtd/yp5eJ9yyqA2tl54htGjVjRoIbUabru/fJJtagcAOCECPACgxixOzVLFGnYhfl7mFgPnsWeJdGid5OEjdR1z2lMqhtEv2HZYhSXltVgc6q2Oo2xbyeUfllZ9bHY1AHBaBHgAQI1ZtP2w2SXAGS192/a1w/WSf8RpT2kXF6y4EF8VlpZr0Q6G0aMWePpIF99tu79jhnRwrbn1AMBpEOABADXCMAwt2kGAx59kpdrCkSQljz3jaRaLRYNP9ML/vJFh9KglMR2kFkNt95e+KZXkm1sPAPwJAR4AUCO2pR9X5vFieXvwqwYnWT7J9rX5YCmi2VlPHdo+RpI0b2uGikoZRo9a0uUWKSD6xFD6j8yuBgAq4a8qAECNWHhi+Hy7uGCTK4HTyD8irZtiu3/xuHOe3jE+RHEhviooKdfC7QyjRy3x9JV6TZBkkXbOlvavNLsiALAjwAMAakTF9l9dE8JMrgROY9VHUlmRFNNRatzznKczjB6miWortbrcdn/pW1Jxnrn1AMAJBHgAgMPlFJRq9V7b/u9dE0JNrgZOobRI+u0D2/2Lx0sWy3m9jGH0ME3nUVJQnFR4VPrtfbOrAQBJkofZBQAA6p5FOw+r3GqoeVSAIoN8zC4H1ZSa6bhex9BtXyou/7AU1FBqfcV5v65iGP2B7EIt3J6pQW1jHFYTcFYePlLPCdLMB6XfF9hGjTTqYXZVAOo5AjwAwOEqhs+ntIw0uRJUh4+nbYDehK/WOahFQ7O93pDcpCPt/qJwd8/zfmXFMPoPF6fp543pBHjUrshWUpurpE3/k5a9LUW2lnyCzK4KQD1GgAcAOFS51bAvONavBQHeFcUE++rVazuoqNTqmPYO/6rmqw7ouOGrjKbXKryKrx/SPkYfLk6zD6P38XR3SF3Aeel4o7RvpZSzV1rxntTnH2ZXBKAeI8ADABxq3b5jOlZQqiAfD3VpHKpt6cfNLgnVEBPs67C2Wq39XJL0ZXmKkr2q3nvZqdIw+sMadGJhO6BWuHtJve6Vpt8v7f5FanyxlNDL7KoA1FMsYgcAcKj5J4bP927eQB7u/Jqp7/yOblFI+lJZLe6aXDawWm1UXo3+kCPLA85PRDOp3f/Z7i9/RyrMNrUcAPUXPfCAC0rLyld+cZnD2/X39lBihL/D20X9Mn+bbf/3fsx/h6TYrR9JkvZFX6oDaQ2q3Q7D6GG69iOlfSukY7ul5ZOkvo+c924KAOAoBHjAxaRl5Svl5YU11v6Cv/clxKPaDuUUauuhXFksUp/m1Q9rqBu8CtIVnvaTJGlrws1SmlHtthhGD9O5e0q97pN+vk/au8y2Mn1SP7OrAlDPEOABF1PR8z42paniQhw3R/VAdqEmLUitkZ591B8LTvS+d4wPUXiAt8nVwGzR2z6Rm1Gm3MiuOhrSVtLGard18mr00zceIsDDHGFNpA43SGs/tS1oF91O8ufDSgC1hwAPuKi4EF96yuF0Kua/s/o83ErzFbXzv5Kkg63/6pA2K4bRz2UYPczUdoS0f4V0eLu05HXp0mclC+t9AKgd/N8GAOAQRaXlWpKaJYn93yFFpn4jj5JcFQYm6FjD/g5ps2IYfUFJuRZuP+yQNoEqc3OXet4nuXtLh9ZL2342uyIA9QgBHgDgEMt/P6LC0nJFBXmrTWzVtwpDHWItU8y2jyVJh1r9xWG9kyevRj+d1ehhpuA46aJbbfdXT5Zy9ptaDoD6gwAPAHCIBRXD51tGysLKzPVa+J4Z8snbp1LvUB1OGuHQtoe0j5Ek+2r0gGlaDJFiOknlxdLiVyUrP48Aah4BHgBwwQzD0NyttgCfwvz3+s0wFLf5PUlSesubZfVw3GKbkm0YfWywj/JLyrVoB8PoYSKLm9TzHsnTX8raIW361uyKANQDBHgAwAXbln5cB7IL5e3hpkuasSJzfRZ86Ff5H9uqcndfpbcY5fD2LRaLhrSz9cL/vIFh9DCZf4TU/W+2++umSEd2mVsPgDqPAA8AuGBztmRIki5pFiFfL1YGr8/iNr0vScpoNlJl3qE1co1Kw+jLqr+3POAQTfpKjS+WjHJp8StSeYnZFQGowwjwAIALNnerLcBf2jrK5EpgpoCs9QrOWCarxUOHWo+psetUGka/p6jGrgOcF4tF6jFW8gmRsvdKaz83uyIAdRgBHgBwQdJzirRhf44sFqlfSwJ8fRZ7Yu57VuIVKvGPrbHrnDyMfnpqYY1dBzhvPsHSxeNt9zd/L2VsMrceAHUWAR4AcEEqet87xYeoQaC3ydXALD45vyts72xJ0sE2t9f49SqG0c/9vUhFhmeNXw84p/juUtNLJRnS4tek0gKzKwJQBxHgAQAXpGL++wCGz9drsVs+kEWGjjYcoMKQZjV+Pfsw+lJDi6wdavx6wHnpepvkHynlZUgrPzK7GgB1kKkB/pdfftHw4cMVGxsri8WiqVOnVnreMAw98cQTiomJka+vrwYMGKCdO3dWOufo0aO68cYbFRQUpJCQEI0ZM0Z5eXm1+C4AoP7KKy7Tsl1HJEmXEeDrLa+CdDX4/XtJ0oE2d5z95MPbpYPrLvhmObReQ5p4SJKmlfdw9FsCqsfLT+o1wXZ/5yxp/0pTywFQ93iYefH8/Hx16NBBf/nLX3T11Vef8vy//vUvvfnmm/rkk0+UmJioxx9/XAMHDtSWLVvk4+MjSbrxxht16NAhzZkzR6Wlpbr11lt1++23a8qUKbX9dgCg3vllx2GVlFuVEO6npAYBZpcDk8Rs/Vhu1lLlRnZVXmSX05+Ul2n7+t1tkttuh1x3mDVJH+pZzbV2VoGlVH4OaRW4QNHtpdZXSFt+kJa+KV0+SfIJMrsqAHWEqQF+8ODBGjx48GmfMwxDr7/+uh577DFdccUVkqRPP/1UUVFRmjp1qkaOHKmtW7dq5syZWrlypS666CJJ0ltvvaUhQ4bo5ZdfVmxszS2gAwCQ5m75Y/V5i8VicjUwg3txjqJ22D40P2vve1mx7Wunm6T4CIdcu4Mhxc8s1758H80/FqphYQ5pFrhwnUZLB9ZIOfukFe9KfR40uyIAdYSpAf5s0tLSlJ6ergEDBtiPBQcHq3v37lq2bJlGjhypZcuWKSQkxB7eJWnAgAFyc3PTihUrdNVVV5227eLiYhUXF9sf5+bm1twbAYA6qqzcqvnbbb2qA1oxfL6+it7xudzL8lUQ0lzZcSnnfkFAtBSe4JBrWyQNb1akd9aV6KfUUg1LYjE7OAkPb+mS+6Wf75d2/yo16iEl9jG7KgB1gNMuYpeeni5Jioqq/EdhVFSU/bn09HRFRkZWet7Dw0NhYWH2c05n4sSJCg4Ott/i4+MdXD0A1H2r9hxTdkGpQv081aVxqNnlwARuZUWK2TpZknSgzd9s+2HXsuFNbaF9wb4yHS8xav36wBmFN5XaX2e7v/xdKT/L3HoA1AlOG+Br0sMPP6ycnBz7bd++fWaXBAAup2L1+ZSWkfJwr5e/Tuq9Bru+lWfxERX5x+lIwlBTamgZ5qakEDeVlEtzdpeaUgNwRu2vlcKbSSV50vJJksGHTAAujNP+xRUdHS1JysjIqHQ8IyPD/lx0dLQyMzMrPV9WVqajR4/azzkdb29vBQUFVboBAM6fYRj2/d9Zfb6espYpdsuHkqRDrf8qw82c4esWi0XDTwyd/2lXmSk1AGfk5iH1utf2df9K6fcFZlcEwMU5bYBPTExUdHS05s2bZz+Wm5urFStWKDk5WZKUnJys7OxsrV692n7O/PnzZbVa1b1791qvGQDqi23px7XnSIG8Pdx0SbMGZpcDE4TvmSGfvL0q9Q5TZtNrTa1lWJJtSZ9f95fpWJHV1FqAU4Q0kjrcYLv/2wdSwVFz6wHg0kwN8Hl5eVq3bp3WrVsnybZw3bp167R3715ZLBZNmDBBzz33nH788Udt3LhRo0ePVmxsrK688kpJUqtWrTRo0CDddttt+u2337RkyRKNGzdOI0eOZAV6AKhBMzbZ1hnp3byB/L2ddj1U1BTDUNzm9yVJ6S1Hy+rha2o5TUPd1TrcTWVWaWYavfBwQm2vlsKSTgylf4eh9ACqzdQAv2rVKnXq1EmdOnWSJN13333q1KmTnnjiCUnSP/7xD40fP1633367unbtqry8PM2cOdO+B7wkffHFF2rZsqX69++vIUOGqFevXvrggw9MeT8AUF/M3HRIkjSozZmnK6HuCj70q/yPbVG5h5/SW4wyuxxJfyxm91Mq8+DhhNw8pJ4TJIu7tG+5dHCt2RUBcFGmdpv07dtXxlk+gbRYLHrmmWf0zDPPnPGcsLAwTZkypSbKAwCcxq7DedqRkScPNwvbx9VHhqGGG9+WJGU0vU5l3s6xA8GwJE+9uKJYyw6WKzPfqkh/p50liPoqLNG2qN36/0qb/md2NQBcFOMeAcBFpWXlK7/Y8cOF/b09lBjhf8bnZ54YPn9x0wgF+7Hvdn0TlLFCQZmrZHXz0qE2t5ldjl18oJs6RbprbWa5pqeV6Za2XmaXBJyq3bXS3mXSsd1mVwLARRHgAcAFpWXlK+XlhTXW/oK/9z1jiJ9xYvj84LYMn6+PGm58S5KU2fT/VOLnXD8Dw5t6aG1muX5KLSXAwzm5e9qG0k+7V5Ih7V0uxXY0uSgAroQADwAuqKLnfWxKU8WFOG4BsQPZhZq0IPWMPfv7jhZo04FcuVnYPq4+CshcreD0ZbJaPHSgzd/MLscuLadc+SVSUrCbLJJWZ5Rr/p5SRfpd2DB6fy8pMdjdMUUCFcKbSk36SL8vlBa/JnUeJXmdedQTAJyMAA8ALiwuxPesw90dbdZm2/D5bolhCg/wrrXrwjlUzH0/nHS1SgLiTK7GJi2nXClf5p9y/C8zCx3S/oKR/oR4OF7zwbYAn5cuLXhBGvi82RUBcBEEeADAeavYPm5w2xiTK0Ft889ar9CDi2RY3HWg7Z3Vbif1uLt0uNxhdaVm2/Z9H9vJS3EBblpxqEw/pJYpLsCisZ2q/yHTgTyrJq0tUX6JoyoFTuJx0s/m8ndti9vFdDCvHgAugwAPADgvGblFWr3nmCRpINvH1TsNN06SJB1OvELFgY2r/HofT4skacJvwdJvp/aYX6ikEDfFBLgp3NdTP+0q04E8Qz4eUkwAq9HDiTXpa+uJ/+ke6a/zJDdGewA4OwI8AOC8VAyf79QoRNHBPiZXg9rkd3SrwvbPlSGLDrS9q1ptxAS461WPSSpqN0oKiHRofScH9SBvi9pGuGnDYauWHSzX1c0J8HBiF98tHVhr2xf+tw+kHtUf3QKgfiDAAwDOy4yNFcPn6X2vbypWnj+SMFRFwU2q3U6M5agUWC4F12yovjjOQxsOl2jZwTJd3ZytDuHE/MKlAU9KP98nzX9OajVcCm5odlUAnBgfSwMAzulIXrFWpB2RxPz3+sbv6BaF750pQxbtbzvW7HLOy0XR7vJwk/YfN7Qv12p2OcDZdblViu8uleRJ0x+QDMPsigA4MQI8AOCcZm5Ol9WQ2sYFKT7Mz+xyUIvi178hydb7XhjawuRqzo+/p0UdGtjmEi89ePotEQGn4eYmDXtdcvOQtk+Xtv5kdkUAnBgBHgBwTj+tPyhJGt4+1uRKUJv8j2xU2P45Mixu2tf+HrPLqZLkOFuAX3agXAY9mnB2Ua2lnif+G5vxD6kox9x6ADgtAjwA4KzSc4q0Iu2oJGloe4bP1yfx61+TJGUlXqGi4CSTq6maLlHu8nKTMgoMpeUQ4OECej8ghTWRjh+S5j1rdjUAnBQBHgBwVj9vPCTDkLo0DlXDUIbP1xcBh9cq9MBCGRZ37W833uxyqszHw6LO0Sd64RlGD1fg6SsNs31oppUfSvtWmlsPAKdEgAcAnNUfw+fpfa9PKnrfDze5WkVBCeYWU00Xx/4xjN7KMHq4giZ9pfYjJRm2veHLS82uCICTIcADAM5o39ECrduXLTeLNIQAX28EZvymkEOLZbV4aH971+t9r9Ah0l2+HtKRIkM7j7EaPVzEwOcl3zApc7O04n2zqwHgZNgHHgBwRj+e6H1PTgpXZKCPydWgVhiG4te/LknKbDhQxcVFUnHqhbebs+/C26giL3eLLop216/7y7X0QLlahLnXeg1AlflHSJc+Lf04Xlr4T6ntCCmID1AB2BDgAQBnxOrz9U/wocUKzlguq2HRgd3bpd0OXn3ew9ux7Z1DcqyHft1fruUHyzS6jafc3Sy1en2gWjreJK3+RDqwSprzuDTiQ7MrAuAkCPAAgNPamXFc29KPy8PNokFto80uB7XBsKrR2n9JktKNUJW0v1Hyj3Rc+x7ett7FWtSugZsCPKXcEmnrEavaNqAXHi7AzU0a+rL0QYq08Rup881S4iVmVwXACRDgAQCnVdH73rt5A4X4eZlcDWpD+O5pCji6WWUefjpQHG4L78FxZpd1QTzcLOoW4675e8u19GAZAR6uI7aTdNFfpFUfSdMfkP72q+TuaXZVAEzGInYAgFMYhqGfNhySJF3egeHz9YGlvESN1r0qSTrY5FqV1aHP+JPjbO9l5aFylVlZjR4upN9jtgXtDm9lQTsAkuiBBwCcxsLth5WWlS8vdzfFhvho04GcareVmpnnwMpQUyJTv5RP3l6V+DbQoYSrpK0rzC7JYVqHuynYW8opljYetqpTFL3wcBF+YSxoB6ASAjwAwM7H0zYw65U5OyRJJeVWXfv+coe2DefjVpqvhhvekiTtbzdeVg9fkytyLDeLRT1iPTQrrUzLDpYR4OFaWNAOwEkI8AAAu5hgX710TXs9/dMW5RWXaVSPxmoVE3TB7fp4uikmuG6FwrokdutH8io6osLAxspsdp10bI/ZJTncxbHumpVWplXp5SopN+Tlzmr0cBF/XtCuyy1SQi+zqwJgEgI8AKCSzOPFyisuU6CPhy5rHSUPd3rO6zKPoiOK2WLr0dvX8e8y3OrmIllNQ90U4WtRVqGhdZnl6hbDn0BwIbGdpItulVb9R/r57yxoB9Rj/FUGAKjklx2HJUk9kyII7/VA/PrX5FGap7ywtjrSeLDZ5dQY2zB629D5pQfKTa4GqIZ+j7OgHQACPADgD3nFZVqz95gk2/ZxqNt8j21X1M4vJUm7L3pMstTtPwuSY2297mszylVYxmr0cDF+YdKAp2z3F/5Tyj1kajkAzFG3f1MDAKpk+e9HVFpuKD7UVwnhfmaXg5pkGEpY/bwshlVHGg3S8ahuZldU4xKDLYr2t6jEKq1OpxceLqjTKCmui1Ry3LagHYB6hwAPALCrGD7fu3kDWSws8lWXhRxYoJBDi2V189Kezg+ZXU6tsFgsuvjEMPolB8pMrgaoBjc3aegrkiy2Be12Lza7IgC1jAAPAJAkHcou1M7MPFksUs+mEWaXgxpksZYqYfULkqRDrW5RcWAjkyuqPT0b2obRbzhsVW4xw+jhgioWtJNsC9qVl5pbD4BaRYAHAEiSftmZJUlqHxesUD8vk6tBTYra/oV8c39XiU+4DrQda3Y5tSo2wE2JwW6yGtLyg/TCw0WdvKDdSvaFB+oTAjwAQFbD0OLUP4bPo+7yKM5W/IY3JEn7Otyncq9AkyuqfT3jKobRMw8eLsovTOr/hO3+ghekvExz6wFQawjwAABtPZSrrLwS+Xm566LGYWaXgxoUv+5VeZTkKD+0pTKbXmt2OaZIjnOXRdKOY1ZlFljNLgeons6jpZiOUnGuNPcps6sBUEsI8AAA++J1PZqEy8uDXw11lf+RjYra8YUkafdFj0tu7iZXZI4wHze1ibD9nLMnPFyWm7s05GXb/XVfSPtWmlsPgFrBX2kAUM/lF5dp+e9HJUl9GD5fdxlWJf72pCwylJUwXLnRyWZXZKqL42yL2S3eXybDYDE7uKj4rlLHm2z3p/9dsvKBFFDXEeABoJ77dWeWSsqtig/1VbPIALPLQQ2JTP1agVnrVOYZoN1dHjW7HNN1j3GXp5t0IM/QnlwCPFzYgCcl72Dp0DppzadmVwOghnmYXQAAwDyGYWj+tgxJUv9WUez97opyD0ilhWc9xaMkV41WT5Qk7Wt6k0oLc6XC3DO/IGefIyt0Sn6eFnWKctdvh8q15ECZEoLZeQEuKiBSSnlEmvmgNO9pqfUVtkXuANRJBHgAqMd2ZuZp37FCebm7qRd7v7ue3APSd7ef87RGlkPydDuufMNb6ZsWSpsWnV/7Ht4XVp+T6xlnC/BLD5Tr+laG3PgAC66q61+lNZ9ImVuk+c9Jw141uyIANYQADwD12Nyttt735KRw+XvzK8HlVPS8t79W8o887SkBeWmK3PKKJCmt9V1SYNPza9vDW/Kv2x/qdIx0l5+HdLTI0LYjVrWOqJ+L+qEOcPeQhrwkTR4qrfqPbYX62I5mVwWgBvDXGgDUU3nFZVr++xFJUv+Wpw9/cBH+kVJw3KnHreVK3PqKLDKUGZOi4w371H5tTszL3aLuse5asNc2jJ4AD5eW0EtqO0La9D9pxj+kv8ySGFUC1DksYgcA9dTinYdVWm6oUZifmrJ4XZ0Uu/cHBRz/XWUeAdrT/Gazy3FKFavRrzhUrtJyFrODi7v0WcnTX9q3QtrwldnVAKgBBHgAqIcMw9DcrZmSbL3vLF5X9/gUHFL8ri8lSbtb/EVlXiHmFuSkWoe7KdTHovxSaf1htuCCiwuOk/o8YLs/+3Gp6CyLVQJwSQR4AKiHdmTk6UB2obw93NSrWd2e51wvGYaabH1HbtYSZYd10OGYFLMrclpuFosujrUNnV+8nwCP/2/vvsOjqvI/jr/v9PTeCQECoXekKU1xURS7orCKrIoFdRX5ubq6a1n7KmLDLqiLYgMbiAhSBaR3SEhID6mQnun398dAFEEgMMnNJN/X89wnkzszdz7hkMx87zn3nBZg8F0Q0RFqimHl81qnEUJ4mRTwQgjRCi07Onldhwj8TTIdSksTVfAzIYd24tKZOND1TrkO9hTOPTKMfkuRi1qHDKMXPs5ghouPFO7r34TivdrmEUJ4lRTwQgjRylTWOVifeWTyuq4yeV1LY7Qdpl3abABykydg84/VOFHz1y5EIT5QweGGjYXSCy9agI6joculoLo8E9qpcmJKiJZCCnghhGhllu4twuFSSY4KIDlKJq9radqlvovBWU11UAcOth2ndRyfoChKfS/8L3lOjdMI4SVjngaDBTJXwZ6vtU4jhPASKeCFEKIVcbrc/LTHM3z+4h5xMnldCxNRuJrIorWoio6MbneDTpZFO13nJnj+rXaVuim3Sm+laAHC2sF593tu//gI2Gs0jSOE8A4p4IUQohVZd6CM8joHYf5GBrUP1zqO8CKTtYQOe98CIK/9tdQGd9A4kW+JCdDRKUyHCqwrkF540UKc+3cIbQuV+bD6Ja3TCCG8QAp4IYRoJVRV5YddhQD8pVssBr28BbQYqpvk3a9hcNZQFdyJ/PbXap3IJx3thf8lX66DFy2E0Q/GPOu5vfY1KMvQNo8Q4qzJpzchhGglUouqyCytwahXOF8mr2tRYotWEHpoBy6dmfQe96HqZGWBMzE43oBOgYxyN6V1bq3jCOEdXS6B5AvAZYfFD2udRghxluQdXgghWomjve/DOkURbDFqnEZ4ix9WknK/ASA7ZTLWgASNE/muELNCz0gd20vcbC+WAl40gdI07x7PHAQRycfuUxTPsnKzhsD+HyH1B+h8sXdfVwjRZKSAF0KIVqCkysrGrEMAXNRdlhVrKRSXnU66AnSqk8ORAyhqM0brSD7v3DYGtpfY2VYsw+hFIzL6eb7Ov837x75ny/FFfGQnGDIVfpkJix6E9iPA5O/91xZCNDop4IUQohX4cXcRqgo9E0JIDJcPbS1F4v6PCVBsOAyBZHSb6ulpE2flnFg9Zj2UyUz0ojEFJ8CVb4OjznvHrMj1TFRnqzrx/SMehF1fQUUOrHoBRj/uvdcWQjQZKeCFEKKFq7O7WJ5aDMDFPaT3vaUIKtpA/IHPAchoPwGHOUzjRC2DxaAwME7P6jzpgReNLLiJL3cxBcDFL8C8GzwT2vW6HqK7NG0GIcRZk0nshBCihVu6t4hau4u4EAu9E0O1jiO8QG+vouMvD6CgUuwO4XBYb60jtSjD2/zWv+FwSU+8aEG6jIXOY8HthIXTQJX/30L4GinghRDHcbjcFFZY2ZVfwfbccooqrbjc8ibvi+xONwt3HgTgst7x6GSIdYvQbuOTWGrysfrFkqnGaB2nxekWqSPE5Lm9oVB64kULc/HzYPSH7F9g+6dapxFCNJAMoReilXOrKgdKauqHWN/w7nqqrM7jHqfXKUQFmmkT5sfQjpGM6hxFrzah6HVSEDZny1OLqahzEBlo4rxOkVrHEV4Qnv0D0Qe+QlV0pPf+B+5f3tE6UoujUxT6ROtZmefi5xwnt/cxax1JCO8Jbeu5Hn7p47DkUUi5CPzDtU4lhDhNUsAL0Upll9WweFchW3LLqaxz1O8/WrzrdQrhASb0ikJJtQ2XW6Ww0kphpZVN2Yd5ddl+wgNMjEiJ4pr+bRiaHIEivbvNitPl5rvtBQCM6x2PQSeDrnydsbaYDr8+CkB+9zuoCu+ucaKW62gBv6nQRVmdmwg/+f0RLciQu2H7Z1CyF5Y9AeNe0TqREOI0SQEvRCuTc6iWr7bksSHzUP0+P6Oe5KgAdhVU8saEvgxJjiTUz4juSO+6y61SWm3jYIWVtKIqVqaWsCqthEM1dhZszWfB1ny6xQVz2/D2XNorHqNePug2B6v3l1JWYyfUz8jIlGit44izpaokr/sHRtthqsN7kNfrXijP0TpVixUT4Pk75lLh23QHk3tKL7xoQfRGuHQGzL4YNs+BPhMhcaDWqYQQp0EKeCFaiaJKK59uyOHXI4W7AgzuEMH5XaLpEhtE7uE6/rlgJ0kRAYQHmI55rl6nEBNsISbYQp/EUK4bkIjD5WZL9mG+33GQLzfnsedgJfd/tp0XFqdyx4hkJg5qi0EKec243CrfHul9v6RXHCaDtIWvi0n7H2EFK3HrzaSfNwNVbzr1k4RXzE+TAl60QElDPYX7trnw/TSYsgL0UhoI0dzJb6kQLZyqqqxILeHDdVnYnG4ABrUP5+p+bc5qPXCjXsegDhEM6hDBtAtTmPtrNnPWZnOwwspj3+7m0w05PH5ZdwZ3iPDWjyIaYP2BMgorrQSaDYzuKpOc+TpLxQGSNj8LQHa/h6gL6ahxotZDr8DOUjdph1ykhOu1jiOEd134JOxbCEU7YcPbMGSq1omEEKcgXTJCtGBVVgcvL03jndUHsDnddI0L4vmre3Hf6JSzKt7/KCzAxN3nd2LNP0bxn8u7E+pvZF9hFde/s557P91KUaXVa68lTs2tqny9LR/wrPtuMUrR4csUt4NOv9yP3mWlPG4YhZ1v1DpSq3JOrOf3Z/5+xykeKYQPCoj0FPEAy5+Binxt8wghTkkKeCFaqF35FTz41Q42Zh1Gr1O4YWBbHh3bjbZeLNz/yGLUc+OQdix/YCQTB7VFUeDb7QVc8NJK5m/JQ5X1ZpvEpqzD5B2uw8+oZ0z3WK3jiLPUZsfrBJbtxGkKIWPo86DIW3dTOr+tZ7DigjSHLKcpWqa+N0KbgWCvhsUPaZ1GCHEK8ilAiBZoRWoxz/6wl/JaB/GhFv5zeQ/PGuBNtORbWICJp6/syXd3n0fftqFU25xM+3w7987bRkWd9GI1Jpdb5fNNuQCM6R5LgFmulPJlgSVbSNj1BgAZg5/G7i8nZJraObF6Qs0KRbUqq/NkTXjRAul0cOnLoOhh77eQtkTrREKIk5ACXogWRFVVvt6az9urDuBW4byOkTxzZU/aRwZokqdHQghf3jGUBy5MQa9T+G57AWNfWc2vB8o0ydMarEorIb+8jkCzgXG947SOI86CzlFDp18eQFHdlLS/gkNJY7WO1CoZ9QpXdDIC8EWqXeM0QjSS2B4w+E7P7UXTwV6jbR4hxJ+SAl6IFsLtVpmzNovPjvS+XtY7nrtGJmM2aHv9s16ncM8FnfjyjiEkRfiTX17H9e+u543l6TKk3stsThdfbskD4Mq+CfibpPfdl7Xb9DSWqmxsAfFkDnxc6zit2rWdPQX8kiwnh+rcGqcRopGMfBiC20B5Nqx4Vus0Qog/IQW8EC2A0+3m1Z/3s2RPEQowaUgSNwxsi6I0zZD509G3bRgL7x3G1f3aoKrw3x9TmfrJFmpsTq2jtRg/7irkUI2dyECTzDzv48JylxKTPg8VhfRzX8JlCtY6UqvWPVJP90gdDjd8ky6XAYkWyhwIl7zkub3uDSjYqm0eIcQJSQEvhI9zu1Vmrcjg18xDGHQK95zfiYt6NM+h04FmAy9d15tnruyJUa+waGchV81aS3aZDNU7W9VWJ98cWff92v6Jsu67DzPWlZC87mEADna7lcqYQRonEgDXdTYB8Nk+h4weEi1X54ugx9WguuHbe8AlJ6yEaG7kE54QPkxVVT74JZN1GWXoFYVpF6YwJLn5r7s+YVBb5k0ZTFSQmdSiKi57/RfW7C/VOpZP+2Z7PrV2F4nh/pzXMVLrOOJMqSod1v8To62MmrAu5PSZpnUiccTlHY2YdLDvkJvdpTKMXrRgFz0PfmFQuNPTEy+EaFakgBfCh83bmMuyfcUowNRRyfRtG6Z1pNPWPymc7+/xzFJfUefg5tkb+GpzntaxfFJptY0fdxcCcMM5iU222oDwvuj0zwjPW4ZbZyL93BmoerPWkcQRoRaFv7T3zCvxuUxmJ1qywCj4y9Oe2yuehbIMbfMIIY4hBbwQPmplWgnfHhkyfcuw9gxJ9r1e15hgC/OmDObyPvE43SoPfLGd15btl+GpDfT5xlwcLpWucUH0SQzVOo74M5X5UJb+p5slZyXtNj4JQE7KzdS6DSd9PGXpUJGr8Q/VuhwdRv/1fgdWp/ydEi1YnwnQfgQ4rfDd30Hel4VoNmSKYiF81NEe1wkD23JBF9+dsMxs0PPydX2ID/XjzRUZvPRTGvnldTx1RQ8MejnHeCqZpTWsTi9FASYMTGpWExeK36nMh/lTTvIAlY66bPSKlQrVn4O7V8Hu1ad/fIP01DeFcxP0xAcqFFSrLMlycllHo9aRhGgcigLjZsKsoZC1GrZ8CP1v1jqVEAIp4IXwOXsPVtbfvqx3PON6x2uYxjt0OoV/XNSF+BALj327m3kbcymqtPL6hH4EmOXP1Ml8sy0fgPO7RNMxOlDjNOJPOeo8X3tdBwHRx92dkP8DQfn7cOr9SO/xMJjDT//YBjME+N4IHF+k1ylck2Lk1S12vki1SwEvWrbwDnD+o7DkEfjxUeg4GkLaaJ1KiFZPureE8CEF5XU8vWgvAN3ighl/TqLGibzrxiHteOuv/bEYdSxPLeH6d9ZTXGXVOlazVlxlI8hi4Ppz2modRZyOgGgISThmC6CWxIJFAGR2vQN7dM/jHnPSTYr3JnXNkWH0a/Jc5FXJZHaihRt8J7QZCPYqGUovRDMhXVtC+Ig6u4spH2+ivNazpMu1A9qga4Th0unF1V49XoDZQPvIgNN+/F+6x/LpbYO55cNN7Myv4KpZa5kzeaD0Lv/B709sTBzUlkCL/Dn3RTqXjU67ZqKobkpjzqM0drjWkcQptA3WMSRez7oCF5/vszPtHIvWkYRoPDo9XP4GvHUepC+FbXOh71+1TiVEqyaf+ITwAaqq8n9fbmdXfiXBFgOVVidmg96rr2Exegbk3PfZNq8eF2D59JENKuL7tg1j/p1DmTR7A9lltVz95lremzSAc9o1YFhxC/fuqkwA2kX4M7xTlMZpxJlKSpuDX20+NnMEmV1v91x3Kpq9CV1NrCuo47N9Du7tb8YgKz+IliwqBc5/BH76Nyz+JySfD8G+f/meEL5KCnghfMCsFRl8v+MgBp3CP8d25aH5O73+GnEhfsy4rjdWh/eGhOaX1/HG8nS255ZTY3M2+PnPXNmTJ7/bQ2pRFRPeXc/0v3Tm3CNrnDe0Z78l+XlfEesOlAFwWZ8EmbjOR4WWbiY27wcAMrrfi9MYpHEicbrGtDcQYVEoqlVZlu1kTHu5Fl60cEPuhj3fQP5m+O4+mPCZnHAUQiNSwAvRzK0/UMZLS1IBePLyHvRICGm014oL8fPq8bzZq+9wqTz7w75j9jW0Z78lqLI6+NfXu+u/jw2W4bu+yGCvJHn36wAcbHspFRG9NU4kGsKkV7ims5G3t9v5ZK9dCnjR8un0cPkseHsY7P8Rtn/qWWpOCNHkpIAXohkrrbZx76dbcatwdb82TBjUll35FVrHOm3e6tV3uVW+217AhqxDAPRrG8qWnDPr1fd1j327m/zyOmKCzRRV2rSOI86EqtJh7yxM9sPUBiSS0/FGrROJMzChq4m3t9tZlesit9JNYrDMCyxauOguMPJhWPYE/PAPaDcMQlvWZLpC+AIp4IVoptxulfs/20ZxlY2O0YH854ruWkc6I97q1b9vdCcWbM3ni815bMkpB8Dhal0zQH+3vYD5W/LRKfDAhZ158KsdWkcSZyDq4HIiitfjVgzs73E/br2s4e6LkkJ0DGujZ3Wei0/32nlwkIyGEa3A0Hsh9QfI2wBf3wk3fQs6OXklRFOS3zghmqlZK9JZvb8Ui1HHrIn98De17vNtiqJwVb823D68A0fni3ry+z1Ut5Je+PzyOh5Z4Jn74O5RHekWH6xxInEmzLZS2u17F4Dc5BuoDe6gcSJxNiZ09Swp93mqA4dLltcSrYDeAFe+BUZ/yFoNv76ldSIhWp3WXREI0UytP1DGjJ/SAPjP5T1IiZHJrY4a2Tkaq8PNh+uy2JpTzvi31zF78jlEB7Xc3i+XW2XaZ9uotDrpnRjKPRd0IrWwSutYosFUOmZ8hMFVR2VoVwraXaF1IHGWRicZiPJXKKlV+SnbydgOci28aAZK07x/THMQRCR7bkckw1+egoXTYOnjnlnpo7t4/zWFECckBbwQzczhGjt/n+e57v2a/m24doBcX/ZHnWM9JzRC/IzsLqjkqllr+fBvA0mOaplrxb+z6gC/Zh7C36TnlfF9MOpl8JQvilfKCK4uwan3I73HfaB4dylI0fSMeoXxnY28vtXOJ3vsUsALbRmPXLI2/7bGOf49W34r4gf8DVIXedaGX3A73LoU9PL/X4imIAW8EM2Iqqo8PH8nRZU2OkQF8OTlvnnde1N58dpePLVwL9lltVxzZK34/kkta634DZmH6lcheHxcd9q1sln3NVOZD446rx0uIH81iUoJAFldbsXmF+O1YwttXd/VxBtb7azJd5FV4aZdiJxgExoJToAr3/bq3y4AKnJh9Utg+93IL0WBy16HWYPh4DZY+Tyc/6h3X1cIcULNuoB//PHHeeKJJ47Z17lzZ/bt8ywlZbVaeeCBB5g3bx42m40xY8Ywa9YsYmLkg5HwTV9szmPx7kKMeoVXr+/b6q97P5W4ED++unMof5uzkR15Fdzwzq88d3VPrurXRutoXpFdVsPtH2/C6Va5tFcc1w5oGT9Xs1eZD/OneO1wOtx00mWiU6Asoj8lced77dhCe22CdIxINLAi18ncPXYeGdJyL+cRPiA4oQlfKw4unQFf/s1T4CefD0lDm+71hWilmn110L17d5YuXVr/vcHwW+T777+fhQsX8sUXXxASEsLdd9/NVVddxS+//KJFVCHOSnZZDU9861nfe9qFnRt1vfeWJDLQzLwpg7lv3jaW7Cli2ufbSSuq5v/GdEZ/dLY7H1RR5+BvczZyuNZBrzYh/Pea3iiK7/48PuVo71Wv6yAg+qwPl5T1KX7FqdhNoRzo8XdPz5VoUW7qbmRFrpPP9tm5f4AZf6O0sWglelwN+5fC9k/gq9vgzjXgF6Z1KiFatGZfwBsMBmJjY4/bX1FRwfvvv88nn3zC+ed7ejNmz55N165dWb9+PYMHD27qqEKcMafLzX2fbaPG7mJg+3CmDJeZqRvC32Tgrb/256WfUnljeQZvrcwgvbiKmdf3JdDc7P/MHcfhcnPX3M1klNQQF2LhvZsG4GeS66WbXEA0hJxdb1ZYyQZii9cAkN7jfpwmWT2gJRrZ1kBSsEJ2pcqC/Q4mdjNpHUmIpjP2BchdD4cOwLf3wnUfyYlKIRpRs79Qa//+/cTHx9OhQwcmTpxITk4OAJs3b8bhcDB69Oj6x3bp0oW2bduybt26kx7TZrNRWVl5zCaEll77OZ2tOeUEWQy8PL6PT/cca0WnU/i/MV2YOb4PJoOOpXuLufKNX0gv9q3Z2lVV5d/f7OaX9DL8TXremzSA6GAZkuuLjLZykne/AUBB28uoiOitcSLRWHSKwqQenqJ9zi47qipLyolWxBwEV78POiPs/Ra2fKh1IiFatGZdwA8aNIg5c+awePFi3nzzTTIzMxk2bBhVVVUUFhZiMpkIDQ095jkxMTEUFhae9LjPPvssISEh9VtioszyLbSzLbec15enA/DUFT1ICPXTOJFvu6JvAp9NGUx0kJn9xdVc9vovfLMtX+tYp0VVVWb8lManG3JQFHj1+r50j5dLKXySqpK85zWMjgpqAtuR0+lGrROJRnZNiokAI+w/7GZtvkvrOEI0rYR+cMG/PLd/eAhKUrXNI0QL1qwL+Isvvphrr72WXr16MWbMGBYtWkR5eTmff/75WR334YcfpqKion7Lzc31UmIhGsbqcPHA59twuVUu6x3P5X2acPKZFqxv2zAW3juMockR1Npd/H3eNv65YCdWR/P9UK2qKk8t3MtrP3tO5vzrkm6M7iYTcvqqmLwfCCvdjFtnZH/Paag6WV6ppQs2K1yT4mnn2bvsGqcRQgND7oEOo8BZ55nYztuz4QshgGZewP9RaGgoKSkppKenExsbi91up7y8/JjHFBUVnfCa+d8zm80EBwcfswmhhRd/TCWjpIaoILMsGedlUUFmPr5lEPee3xFFgU9+zeGqWWvZV9j8LplxuT3LB76/JhOAJy7rzt/Oa69xKnGm/KpzaZc2B4DsTpOoC2yrbSDRZG46Mox+WbaT7Aq3xmmEaGI6HVz5FvhHQtEu+OFBrRMJ0SL5VAFfXV1NRkYGcXFx9O/fH6PRyLJly+rvT01NJScnhyFDhmiYUojTsyHzEO//4inYnr+6J6H+MumRt+l1CtP+0pk5kwcSHmBiz8FKxr22hteW7cfhah4frh0uN/d/to15G3PRKfDCNb2YNLSd1rHEGVLcDjruehmd2055RB8KE8dqHUk0oeRQPSMS9ajAR7ulF160QkGxcPV7gAJbPoJtn2idSIgWp1kX8NOnT2flypVkZWWxdu1arrzySvR6PTfccAMhISHccsstTJs2jeXLl7N582YmT57MkCFDZAZ60ezV2JxM/2I7qgrX9m/D+V1kqHRjGpESxeK/D2N01xgcLpWXfkrjijd+Ye9BbXvjiyqtTPpgA99uL8CgU3j1hr5cN0Dm5PBliRmfEFh1AIcxiPTu94LSrN9mRSO4+Ugv/OepdmocMpmdaIWSR8HIhz23v58GRbu1zSNEC9OsP1nk5eVxww030LlzZ6677joiIiJYv349UVFRALz88stceumlXH311QwfPpzY2Fjmz5+vcWohTu25H/aRc6iW+BAL/xrXTes4rUJ0sIV3b+rPK9f3IdTfyO4CT2/8E9/t5nBN0/eULd1TxEUzV7E2oww/o563b+zPpb3imzyH8J7gQzuJz/oagIxuU3GYw7UNJDQxItFA+xAdVXaYn+bQOo4Q2hj+f5B8vud6+M8ngc23VoQRojlr1gX8vHnzKCgowGazkZeXx7x580hOTq6/32Kx8MYbb3Do0CFqamqYP3/+Ka9/F0Jrv6SX8vH6bABeuKY3wRaZ3KqpKIrC5X0SWHL/cC7qHovTrTL7lyyG/3c5b63MaJJJ7qwOF//+Zhe3frSJw7UOuscH8/2953FBVxmF4csM9ko67pqJgkpRwoUcjpaRYK3V75eU+2CnHZdbeuFFK6TTwVXvQnAClO2Hb+8BWV5RCK9o1gW8EC1NldXBg1/uAOCvg9tyXqdIjRO1TtFBFt66sT8f3zKQrnHBVFmdPPfDPi54aSUfrcui2ub0+ms6XW6+2pzHRTNX8dE6zwmcW89rz/y7hpIcFej11xNNSFVJ3vM6ZlsZdf7xZKX8TetEQmPXdjYSbILMCjc/ZXn/74kQPiEgEq6dAzoD7F4A697QOpEQLYJB6wBCtGSZpTXU/K4YfHXZfvLL64gNtnB57wR25Vc0+JjpxdXejNiqDesUxff3RLJgaz4vLUklv7yOf3+zm+d/2MdV/dpw05AkOsUEndVrOF1uvt5WwOs/7yerrBaAyEAzL17bi5Gdo73xYwiNxeYuJLxkA27FQFqv6bgNflpHEhoLMCrc1N3E61vtvLndxpj2BhRF0TqWEE0vcSCMecYzI/1P/4LortDxAq1TCeHTpIAXopFkltYw6sUVJ7yvsNLKtW+vO6vjW4wygMYb9DqFa/q34dJecXy2MZeP1mWRUVLDx+uz+Xh9Nr3ahHBex0jO6xRJ/6QwzAb9KY9Za3eyLqOMFaklLNtbREGFFYDwABNThnfgxsFJBJjlz29LEFCZQdLRJeNSJlMb1EHbQKLZmNTDxDs77GwvdvPrQReD4+V3XrRSA6dA4Q7Y+j/4cjLcthwikk/9PCHECcm7iRCN5GjP+9RRHQnzN/LKsv1UWZ2cmxzBJWc5WZnFqCMuRHr5vMli1DNpaDtuGpLE2owyPlqXxU97itiRV8GOvApmrcjAz6inZ0IIsSEWYoLNxARbCLYYKa+zU1Ztp6zGTv7hOjbnHMbu/G2ZuvAAE7cP78BfpXBvUXTOOjrteBGd6qQserAsGSeOEeWv49rORubucfDWNrsU8KL1UhS4ZAaUpELeRvj0Brh1KViCtU4mhE+SdxMhGllCqB8Ldx6kyuokPsTClOHJmAzSe95cKYrCuR0jObdjJMWVVlbtL2XN/hLWpJdRWm1jQ9ah0zpOQqgfIztHMbJzNOd1jMTPdOqee+FDVJUOe9/Er+4gNksUGd3u9nxIFeJ3butl5tO9DlbkOtlb5qJrhPwdEK2UwQzj/wfvjITSVFhwO4yf65nsTgjRIFLAC9HIduVX8Et6KYoCd46U4t2XRAdbuKZ/G67p3wZVVdlXWEVaURXFlTaKq6wUVdqotDoI8zcREWAiPNBEZICZfkmhJEcFyjWvLVh0/hKiClehKjrSej6AyygTEYrjtQvRcXF7AwsPOHlnu52Xz5eRU6IVC4r1FO2zL4bURbDsCbjwCa1TCeFzpIAXopF9sy0fgHG94ukYfXYTogntKIpC17hgusbJkL/WLrAijfb73gUgp+NfqQ7tonEi0Zzd0cfMwgNOvk138MA5ZtoEyUlc0Yq16Q+XvQYLpsAvMyG0LZxzi9aphPApUsAL0UjUI+ud1thdJIb7c03/NhonEkKcLYO9gpTtz9df916QdKXWkUQz1zNKz7kJen7Jd/HeDjuPn2vROpIQZ6Y0zTvHieoM/SfD5tmwaDqEtIGUMd45NkBZBtiqvHe8o8xBMvmeaBakgBeikSxPLQFAp8BdI5Mx6qXXRQif5naRsvPFI+u9J5DR/V657l2cltt7m/klv5bP9tm5u5+JSD95PxA+xHjk0o/5t3n/2KobvpgEk3+A+L5nf7yyDHit39kf58/cs0WKeKE5KeCFaAQF5XW8vTIDgPO7xNAuIkDjREKIs9U24xNCDu3EpbeQ2vshXAZ/rSMJHzGsjZ5eUTp2lLh5d7udhwdLL7zwIcEJcOXb4Kjz7nEPZ3mG0Tvq4JPxcMtPEJZ0dsc82vM+7AEISTzbhL+pyIXVLzVOz74QDSQFvBBepqoq//hqBzV2FwAjUqI0TiSEOFvhRWtJyPoKgIxud1MX6MUPhqLFUxSF+/qb+dviOj7abee23tILL3xMcELjHTu8Axw6AP+7ytMTHxh99scMSYSIjmd/HCGaISnghfCy//2aw+r9pZj0OuwuN3qdDLH1JZmlNdTYnF49ZoDZQPtIGYXhqwIq9tNp10wACtpeRlnsedoGEj5pVFsDvaN0bC9x8/Y2O48MkV54IQC46HlYOA3K0uHjK+Hm78EvTOtUQjRbUsAL4UXZZTU8s3AvAJOGJvHu6kyNE4mGyCytYdSLKxrl2Munj5Qi3geZ6krosu0ZdG47hyP7k50ySetIwkcpisJ9A8xM/qGOj/d4euGj/aUXXggCo+Gmb+CDi6BoF/zvGrjpa8+kcUKI40gBL4SXuNwqD3y+nTqHi0HtwxnXO14KeB9ztOd96qiOJIR6Z73m/PI63lie7vVefdH4dC4rXba9gMl+mJrAJNJ6TgdFr3Us4cNGJhroE61nW7GLt7fZ+ddQ6YUXAvBMDHfT1zDnEsjfBJ/eABO/+G0CPSFEPSnghfCS91YfYFP2YQJMel68tjcVdQ6tI4kzlBDqJ73lrZ5KSvoHBFRnYzeFsq/Po7gN8kFSnJ2j18Lf/EMt/9tj5/beJqIDpBdetHK/X55uzLOw8H7IWg0fXQEXPgkG85kdS4gWSgp4IbwgraiKl5Z43jT+dWk3EsP9qciv0DiVEOKMqCrtlSLCKg7j0pnY1+cR7H4yGaXwjhGJevpG69la7OKt7Xb+Lb3worU61fJ0uevhg7+c3bGFaIGkgBfiLDlcbqZ9vg27y82ozlGMP0dmpxbClyWmzSFWdxgVhfQe91ET0knrSKIFURSF+weYuWlRLXOP9MLHSC+8aI1OtjxdaRpsfA9cdojoBOfcevo98Ua/xp01XwiNSQEvxFl6/ed0duVXEuJn5Lmre6EoMuu8EL4qftdbtMn4FIDMduM5FDNU40SiJRrWRs+AWD2bCl28vMnGcyOkt1C0Un9WaEd0hNAkWPY4lO2HLR/CBY+Dyb8p0wnRLMkpXyHOwrbccl5fng7Af67oQUywDIUUwlfFpH5M0tYXAMh2R1MUPUzjRKKlUhSFhwd5ehM/T3WQdsilcSIhmqGY7nDhU2AMgOI98NOjYK3UOpUQmpMCXogzVG1z8vd5W3G5VS7pFce4XnFaRxJCnKHIAwvosOExAPI6TqBAjdA4kWjp+scauKi9AbcKz/5q0zqOEM1TVGcY87RnSbnSNPjhQagq1DqVEJqSIfRCnKEnvt1Ndlkt8SEWnrmipwyd10B6cXWzPp7wDdH7P6PDr48AcLDzJHI7TIDUrRqnEq3BPwaZWZrtZHmOk7X5ToYmyMcyIY4T0REueh6WPgaVebBoOox+3LNfiFZI3imEOAMLdxzki815KAq8PL4PIf5GrSO1KhajZ/DQfZ9ta9Tji5Yvfvc7JG15DoDCTjeQdc6/4NABjVOJ1qJ9iJ6JXY18uNvBM+utfHtVADo5GSzE8ULbwtgXPUX84SxY/DCMfBgS+mmdTIgmJwW8EA1UUF7Hw/N3AHDXyGQGdZChtk0tLsSPGdf1xupwe/3YFqOOuBDvTyglowWaGVUlcduLtNn1JgD53e8gp+//gRRPoond29/M/P0OdpW6+TbdyRWd5ISwECfkH+HpiV/xDBzcDsuegMF3QspFWicToklJAS9EA7jcKvd/to1Kq5PeiaHcNzpF60itVmMU2Y1BRgs0Q24X7Tc+TmzaXACy+z5IQY87NA4lWqsIPx139DHz3w02/rvBykXtDVgMciJJiBMyBXhmo1/7ChxYAeteh0OZMPA20ElZI1oH+Z8uRAPMWp7Or5mH8DfpeWV8H4x6KZ7EyfniaIGWzGArp9PqvxN6cDUqCgcGPUVxyg1axxKt3C09Tfxvt538apUPdtq5q+9prnctRGukN8J5D0BIW9j6MaQuhIocGPEQWEK0TidEo5MCXojTtP5AGS8vTQPgicu60y4yQONEwldIkd08+JWn0WXF7ViqsnHp/Ug/90UOJV2sdSwhsBgU/m+gmWnLrby2xcYVnYzEB8oJYiH+lKJAr+sgLAlWvQiFO2HhNBj1CIR30DqdEI1K3h2EOA1l1Tb+Pm8rbhWu7teGawckah1JCNEAYTlL6PnD1ViqsrEGtGHXRV9K8S6alSs7GTknVk+dE/6z1qp1HCF8Q+Igz+R2QXFQXeSZoX7/Eq1TCdGopIAX4hTcbpX7P99OUaWNjtGB/OeK7lpHEkKcJsVlI2nT03RZeQd6Zw0VsUPYOfZrasO7ah1NiGMoisJ/zrOgV+CHTCcrc51aRxLCN4QlwSUzIKE/uOyw9lVYMxOcciJMtExSwAtxCm+uzGBVWgkWo443JvTD3yRXngjhC/wP7aHXwsuJ3/s+AAe73MyeCz7EaQnXOJkQJ9YlQs/NPUwAPLbGitWpapxICB9hDoILHoO+N4Kig4ylnt74ijytkwnhdVLAC3ESG7MOMeOn36577xwbpHEiIcQpuV3E73qTnj9ciX9FGnZLJHtHvUvWOf+WWYpFs3ffADPR/gpZlW7e3WHXOo4QvkPRQa/xcOFTYAn1rBf//X2QsUzjYEJ4lxTwQvyJ4kord83dgsutcmXfBK6T696FaPYCS7fTc/FVJG39Lzq3g7LEv7B93A+Ut7lA62hCnJYgk8IjQywAvL7FRm6l91ewEKJFi+sF416F2J6eYfRrXobVL4GjVutkQniFdEUIcQJ2p5u75m6hpMpGSkwgT13RA0WRdXmFaK4M1kO03fpfotM/R0HFaQwk65zHKOlwlWe2YiHOUHp50xfQHYIVekXp2FHi5v6f63hsqPmU70EBJmgfom+ihEI0c/7hnp74nV/A9k/gwHIo2QfDH4TITlqnE+KsSAEvxAk8vXAPm7IPE2Q28PaNAwgwy6+KEM2R4rITnT6PtttexmCvAKC4w1Xk9PsHDr8ojdMJX2Y58mf/vp/rNM2xqcjFuAWn13O4/PoAKeKFOEqnh97Xe3rkV/0Xqg7CD/8H/W6Cbld4htwL4YOkKhHiD77anMeH67IBeHl8H9rLeu9CND9uF5FZ35K4fSaW6lwAasK6kjnwCaqiB2gcTrQEcYE6ZoyyYNVwMvgVOU6WZDuxGOC+/maCTSfuhc+vdvPGVjs1csm8EMeL7gbjXoN1r0L2Wtj0ARRsg/PuB78wrdMJ0WBSwAvxO7vyK/jngp0A3HtBJ0Z3i9E4kRDiGKqb8NwlJG57Gf+K/QDYLZHk9bqHok43yCR1wqviArXtoWsbbGR/uZvMCjdLspxMP8ckl3MJcSbMgTDiYdj/I2x4Bwq2wLf3wHnTIKGf1umEaBD5pCPEESVVNm7/eDM2p5tRnaO47wK5RkqI5kJxO4jI/I6E3W/XF+5OUwj53W+nsPNNuI3+GicUwvv0OoU7+5h4eJWVLUUu1uS7GNZGProJcUYUBVIugqiusOoFKM+Gpf+G7ld5lp/TG7VOKMRpkXcBIQCrw8XtH28iv7yOdhH+zBzfF51OejmE0JrOUUN0xpfE73kXc00BAE5jIIVdbqag2624TMEaJxSicSUG67g6xcjnqQ4+3GWnR6SeMIu8PwlxxsKS4JIZnqH0qQth93wo3OmZ4C44Tut0QpySFPCi1VNVlX98tYMtOeUEmPU8fHFXcg/Xknv47I6bXlztnYBCtEKm6nziUj8iOv0zDPZKwDNU/mDXv1GUMkEKd9GqjOtoYGOhk8wKlfd32HlAhtILcXYMZhh8J8T1hrWvQtl++P5eGDwVOozUOp0QJyUFvGj1Xvs5nW+2eXr2amwubv/fZq8e32KUWU6FOC2qSlDJJuL2ziY8dwmK6lm+yxqUREHXWyhJvga3waJxSCGankGncHsfM4+ssrK5yMWybCej28lwXyHOWtJQiOjoWSe+eDesfhEKtsKgO8Dop3U6IU5ICnjRqn2/o4AZP6XVfz91VEcSQr33B9ti1BEXIm8AwsdU5oPDy0tnGf0gOOGEdykuGxHZi4jbO5vAQ7vq95fHDuVg18mUJ4yS5X5Eq5cUrOP6rkbm7nHw0W4HncP1JAbL74UQZy0wGsY8Azs+gx3zIGMZlOz1DKmP6Kh1OiGOIwW8aLU2ZR3igc+3A3B5n3i+2VZAQqifLBsnWrfKfJg/pXGOfdU7xxTxhrpSYvd/Qkzq/zBZSwFw682UtL+Cwi6TqA3r0jg5hPBRYzsY2FXiYnuJm1c223h6mAWzQYbSC3HWdHroMwFie3l64SsLYNF06D8Zul6mdTohjiEFvGiV9hdVccuHm7A53VzQJZq/ndu+fhi9EK3a0Z73XtdBQLR3jllTDDs+rz+2/6E9xO2bTWTmd+jcnoWr7X4xFHb+K0WdbsBpCT/9YzfGaIGKXO8eTwgv0SkKd/Y189DKOvKrVT7e7eDW3iatYwnRcsT2gHGveq6Lz10PG9/1DKnvdoXWyYSoJwW8aHUOVtRx0wcbqKhz0LdtKK9P6EdGiUw4J8QxAqIh5MRD3s+MSmjxBuI3/4uQol/r91ZF9uFgl5s5lHQxqq6B1/Q25mgB8ExyJEQzE2JWuKuvmWfX21iW46RHlI4YfxlKL4TXWIJh1COQugg2vgf5m6A07dTPE6KJSAEvWpWKWgeTPtjAwQoryVEBfDDpHPxMeq1jCdFyuV1Elm4kXpdJwKZHPbsUA4eSLuJgl8lUR/U982M3xmiBowxmCIj07jGF8JKeUXou62jgm3Qn7263c2cf6YUXwqsUBbpcAjHdYeXzv43M2vAOjHtF1owXmpICXrQaVoeLWz/aSFpRNTHBZj66ZRBhAfKhR4jGoHPZiCpYRnzW11isxaCAS+9HUeeJHOwyGXuAF9fa9fpoASGav2s6G9lT5mb/YTf/2+PQOo4QLVNYO7j0Zc8s9TnrYNtcT2/81e957hNCA1LAi1bB5nRx+8eb2Zh1mCCLgQ//NtCrs80LITz0jmpic38gLud7jI4KAByGQA7a/Si88C1csWfR4y6EqGfQKdzX38Qjq60U1aoAqKqqcSohWiCDBXqN9xTwpkDI2whvDYNxM6HH1VqnE62QFPCixbM73Uydu4WVaSX4GfV8cPM5dIkN1jqWEC2KwV5JfPY3xOYuRO+yAmC1RFPQ7gpKArri3vAO1JZDWbr3XlQmmxOtXLifjvsHmHlyrQ2XCp+lOugZLR/thGg0V78Pa2ZA7q/w5d8gYzlc/DyYZAUj0XTkr7xo0ZwuN3+ft5Wle4sxG3S8P2kA57RrwAzXQoiT0jtqiMv5lrjsbzG4PNek1wQmUdDuKkpjzvMszVPjWSKOVS82TgiZbE60YinhnuvhF+x38r89DkYmGhjdTq7PFaJRBMXCzYtg5XOe97StH0POerh2jmcGeyGagBTwosVyuVWmfb6dH3YVYtLreOemAQztKJNSCeENOpeN2JzvSchagMHpWcWhJrAduckTOBx1jmcCoKMCImHYA+C0eT+ITDYnBOfEegp4gPt+ruPrK3V0DJMJWoVoFHoDnP8otB/uWQmlbD+8dwFc8hL0/avW6UQrIAW8aJGcLjf/9+UOvt1egEGnMGtiP0akRGkdSwjfp7qIOriSxPS5mG1lANQGtCE3+QYORQ8B5U+Ws5IiW4hG1z1Cx+4yN5MW1bLgigCiA2R5OSEaTfvhcMcvMP82yFgG30yF7LUw9kUw+WudTrRg8pddtDg2p4upn2xhwdZ8DDqF127oy+huMVrHEsLnhZRto9f6B+i4+1XMtjJslkj2d/8724e8wqGYc/+8eBdCNIl/DrbQLlhHfrXKzT/UUmWXSe2EaFQBETDxSxj1qOc9cNtceG80lO7XOploweTTlmhR6uwubvtoMz/uLsJk0PHWX/tzcU8vLlclRCtkriui87Zn6bblcQKqs3Aa/MnudBNbh86iNH4UKDJUV4jmIMSs8OFYfyL9FPaUublzSS12lxTxQjQqnQ5G/B/c+LVnWdPi3fDOSNj1ldbJRAslQ+hFi1FldXDLnE1syDqEn1HPe5MGcK5c8y7EGVNcduKzvyYh80v0bjuqoqOwzVjyOlyH0yQrOQjR3KSXu+kYquORwWYeXmVlTb6LKT/WMm2AGd3v56VogAATtA+Rk3RCnFKHEXDHavjyFshe45mlPnsdjHlaJlsVXiUFvGgRiqus3DJnEzvzKwiyGJgz+Rz6J8ls80KcqdDy3bTbuQC/uoMAVIR1J7PLFOoCkzROJoT4I8uRT3P3/Vx33H0rcl2syK09q+Mvvz5AinghTkdQLNz0Dax4Bla/BBvfhfxNnlnqw9ppnU60EFLAC5+XXlzNzbM3kHe4jvAAEx/9bSA9EkK0jiWETzLXFtJOl0t42iwA7KYwslNupjR2+LEzywshmo24QB0zRlmwOo/dv7nIyVdpnp0jEvX8JcmA0oDf4/xqN29stVNj92ZaIVo4vQEu+DckDoYFU6BgK7w1HMbNhB5XaZ1OtABSwAuftiHzELd9tImKOgftIvyZM3kg7SIDtI4lhM9RXDbid79Dm51voFPsqOg4mDSOvA7jcRlkNl0hmru4wOOnNWofaiLQpPDhLgcrc11EWBSu7WLSIJ0QrVDKX+D2VZ6h9Hkb4cvJkPEzXPw8mOSzqjhzMomd8Fnf7yjgr+/9SkWdg75tQ/nqzqFSvAtxBoKKNtD7+7G03f4yOredCtWf7T0eJjtlshTvQvi4i9obubG7EYD5+518lebQOJEQrUhoW5j8AwybDiiw9WN4ewQc3K51MuHDpIAXPsftVpnxUxp3f7IVu8vNmO4xfHLrYCICZYIQIRpCb6+i/a//oseS6/GrzMTuF0Van4fZ425LnX+81vGEEF4ytoORid08RfyXqQ7mSxEvRNPRG+GCf8Gk7yAoHsr2w7sXwKr/gst56ucL8QdSwAufUmV1MOXjTby6zLO+5t/Obc+sif3xM8nkOkI0RGjez/T+7iJi0+YCUNRxPNsu+4my+FGAXOsuREtzabKRG7p6ivgvUh3M3WPHrcoSc0I0mfbD4M5foMul4HbAz0/B+6OheK/WyYSPkWvghc84UFLNbR9tIqOkBpNBxzNX9uSa/m20jiWETzFYy2i/8Ukis74DwBrYlozBz1AZN/TII4q1CyeEaFSXdTSiU2DuHgffZzipsKlM6W3CoJOTdkI0Cf9wGP8/2PE5/PB/ngnu3h4Oo/4JQ+7xTIAnxClID7zwCYt3FXL5G7+QUVJDbLCFL24fIsW7EA2hqkQe+Jo+3/6FyKzvUBUdBd1uY/u4H35XvAshWrpLk43c0ceEToHVeS5mbLRhc0pPvBBNRlGg93i461foNAZcdlj6OLwzAnLWa51O+AA5zSOaNavDxbOL9vLhumwABiSF8fDYLuh1CrvyK7z2OunF1V47lhDNjakmnw7rHyWsYCUANWFdyBjyHDURvTROJoTQwohEA0EmhVc22dha7OaZ9TYeOMdMsFl64oVoMsFxMOEz2PYJLHkEinbBB2Og9wS48EkIjNI6oWimpIAXzVZmaQ13f7KF3QWVANw+vAPX9G/DhS+varTXtBhlUIpoQVQ3MWn/I2nLf9E7a3DrTOT1uoeC7lNQdUat0wkhNNQvRs8jQ8y8sMFG2mE3j6y2Mn2gmaRgeR8UoskoCvSdCCkXwbLHYctHsP0TSF0II/4BA24Bo0XrlKKZkQJeNDuqqjJ/Sz7//mYXNXYXYf5GZlzXh1Fdout73aeO6khCqJ9XX9di1BEX4t1jCtGoKvPBUXfCuyzVOSTvfJngw7s9Dw3rTkbP+7EGtoXD2X9+zIrcxkgqhGiGUsL1PHGuhRc32iisUXlsjZU7+5gYFC8fD4VoUgERcNlr0PcmWDgNCnfAj/+EdbNg5D88vfJyfbw4Qv4niGalpMrGPxfs5Kc9RQAMbB/Oq9f3JTbk2LOPCaF+tJc130VrVpkP86cct1vBTYJSRoJShk5Rcak6stUoikpdsPyl0z++QZZlFKI1SAjS8Z/zLLyy2cauUjczN9u5usrNVSkySkeIJpd4DkxZAVv/Byufh8o8+PYe+OUVGPEQdL/CsyydaNWkgBfNxqKdB3lkwU4O1zow6hXuG53C7cM7YNDLcD4hjnO0573XdRAQDUBQ5X46ZH2Kv7UUgMMh3TnQ7nrs5vCGHdtghoBIb6YVQjRjgSaFhwaZmbvHwQ+ZTr5Kc5J+2M3YDlIoCNHkdHroPwl6jYdN78Pql6AsHebfCksfg4FTPPf7hWmdVGhECnihueJKK098v4eFOw4C0DUumBnX9aZrXLDGyYTwAQHRGPyDaZv2ITEFSwGwm0LJ6nwrZTHneq6vE0KIU9DrFG7qYSIpRMf7O+xsL3GTUW7TOpYQrZfRAkOmQt8b4de3YcPbntF3Sx+DlS9AnxugzwSI7yfv9a2MFPBCMy63ytxfs/nv4lSqbE70OoW7RiZzz/mdMBmk112IU1OJLN1Au9yvMTo880MUJowhp9ONuIyBGmcTQviiEYkGOoToeG2Ljdwqz/JyH+y089wICya9FAlCNDlLMIz4Pzj3Xtj5Jax7A4p3w8b3PFtkiqe3vtd1ENpW67SiCUgBLzSxK7+CRxbsZHuep+jo3SaEp6/sSY+EEI2TCeEbLDX5tNflEnpgHwC1AW050O1OqkK7apxMCOHrEoN1PDXMwqytdn496GL+fgd7yly8MNKPXlF6reMJ0ToZzJ4Z6/tMgAMrPNfJ7/seStPg5/94tthekDLGs758Qj/PcHzR4kgBL5pUcZWVGUvS+GxTLqoKQWYDD17UmQmDktDr5My+EKeiuGzE736XNjtfR6fYcStG8pKvoyDpClkaTgjhNSa9wuUdjfx60EWQCfYdcnPFghpu6Wni/gFm/I3yni2EJhQFkkd5Nmsl7P0Wts+DrDWe2esLd8Cq/4J/BLQ7DxIHQ9tBnuL+bCfAK8sAW5V3fo7fMwdBRLL3j9tCSQEvmoTV4eK91QeYtSKDWrsLgHG94/nXJV2JDpb1LYU4JVUlPPdHkjY9g6UmD4ByNYDMXtOxxvTVOJwQoiV7c7Q/X6TZ+Sbdybs77CzOdPDUMD9GJMrHSCE0ZQmGvn/1bNUlkP4TpP0IGT9DbRns+cazARj9Ia4PxPb8bYvqcvrrzJdlwGv9Gu1H4Z4tUsSfJvnLKxpNZmkNFbV2lu0rZt6GXEqqPZPhpMQEctuwDnSNC6a4ykZx1elPkpNeXN1YcYVotvwP76XdxqcIKVoHgM0/luyUyZRtXgCWaI3TCSFaulCLwisX+HNFJwePrLaSW6UyaVEtIxMN/HOwmZRwGaYrxBnzZq92dDfPNvJhqDsEOesg51fIXQ/WCshZ69mOUnSe6+YjOkJEJ4g88jWiIwTHHzs53tGMwx6AkETv5AWoyPXMtN8YPfstlBTwolGkF1cxesaqE96XVlTN/32546yObzHKJHei5TNV55O4fSZRB+ajoOLWmynoNoX87rfjriyAzV9rHVEI0YqMamtkybUGXt5k48PddlbkOlmV5+T6LkbuH2Amyl/em4VokMbs1b5ni6fYBnC7oTQVDm6Hwp1HhtrvhLrDcDjLs6UvPfb5xgBPj3hER892dPi9f4Tne6EZKeCFV9mcLr7ems/Mpfvr9wWYDYxMiWJg+3CMXljT3WLUERfid9bHEaK5MlgPkbDzDWLT5qJz2wEoTRpLTr+HsAW20TidEKI1CzQp/Guohb92N/Lcehs/Zjn5ZK+Db9Id/LWbiVt7maSQF+J0NUav9ol6tHU6iO7q2Xpf79mnqlBdDGX7oXS/Z635snTP7cNZ4Kj57Zr63/vxn2AJgeAETy99cBsIS4Kw9p7iXpa0a3RSwAuvqKh18L9fs5mzNouS3w2Jv7hHLNcNSMRilOF1QpyKwVpG3N4PiE39GIPDc7lIRcwQcvo9SHVkb43TCSHEb9qH6Hl7jD+/Fjh5er2VHSVu3t5uZ84uOzd0NTGlt4n4QCnkRQtTmtY4xwtJbPpebUWBoBjP1u68Y+9z2qE8+9jC/uB2OLjNc7+1wrMV7zn2eaYATyEfmgThR76GJXmuvxdeIwW8OCuphVV88ms2X2zOq5+cLjbYwtiesXzwSxbDOkVJ8S7EKZhqC4nb/S4x+z9F77ICUB3enZy+D1IRd56czRZCNFuD4g18c2UAP+c4eX2Lna3FLubssjN3j52L2huY2M3EoDg9ivwdE77MeGTk5/zbGvf4zYXBBJGdPNtRBdvgnRFw0XNg8IPKfM9WngvlWVCRB/YaKNrl2X4vuM2R46V4tvAOZz8jfismBbxoMKvDxcIdB/lkQw6bsw/X7+8SG8SU4R24tFc8aUVVfPBLlnYhhfABAWW7iE39iMjMb+uHyldH9CSvx1QOJ472TC4jhBDNnKIoXJBk5Py2BtYVuHhti411BS6+y3DyXYaTjqE6JnYzcmUnE6EWKeSFDwpOgCvfBked949t9PMc31cYLEeujf/DjPEuh2f4/tFr6o9udYegMs+zHVjueazO4Ompj0zxDMcHUN1N9zP4OCngxWlxuVV+PVDGN9sKWLTrIFVWJwB6ncKFXWOYOLgt53WMlDPsQpyC4nYQnr2Y2NSPCC7ZXL+/Mvoc8nreLT3uQgifpSgKQxMMDE0wsKvUxdw9dr7Z7yC93M0Ta208vd7GsDYGLk02cGGSkWCz/K0TPsSXimwt6I2envXwDsfut1Z4huKXpv222So9196X/TZnFh+Og8SB0OYcz5bQH/zDm/Zn8BFSwIs/5XS52Zx9mB93F/H9joJjlntLCPXjhoGJXDcgUdZxF+JUVJWAQ7uJOjCfyMzvMNrKAHArBg4lXczBLpOojmrEtVWFEKKJ9YjU8+xwPx4eZOGbdAef7LWzt8zN8hwny3OcmPRWhiUYGNnWwPA2BpJCZMSREF7VWNfrN5QlBNoM8GzgmTyvpthzvJI0KNwOhw6Avdqzfn3Gz789N6TNb0vjRXfz9PrrTrN8NQe12HXlpYAXx6iodbAmvZSle4tYnlpMea2j/r4QPyNje8ZxeZ94BrYLR6eTM+dCnIylMpPwnB+Jyvwa//Lf3vjsflEUdbqBok4TcPjLOu5CiJYr2KxwY3cTN3Y3kX7YxXcZDr7PcJJR7mZZjpNlOZ4Rfe2CdQxro2dAnIF+MXraBCoyqk+IM9Hcr9dXFAiM8Wzthnmuo19w+4kfW5Hn2fYvObPXumdLiyzipYBv5aqsDjZnH+aHXYVszj5MRnE16u/uD7IYGJAUxnmdIunXNqx+Gbg9BytPetz04upGTC1EM+V2EXhoF2G5PxGe+xP+Fb8NDXPrjByKGUpJwoWUR/YHnR7qKj3bmajI9VJoIYRoGh3D9Nw/QM99/VX2HfL0xq/MdbK5yEVWpZusPW4+3uPpOIj2V+gXo6d7pJ7O4Tq6hutJCFLQSVEvxMn52vX6f5bXXgPlOZ7Z8A9neW47ao9/viUEQttBaFtPj31Igmd9+z8updeCSAHfilgdLtKLq9mRV8G23MNsyy1nf3E1qvrnz6myOlmeWsLy1JIzek2LUYbEiRZMVfGrzCC4cC0hB9cSUvQrBntF/d1uxUBlZF/KinIpcwXjyiuGvLnAXO9lMJi9dywhhGgCiqLQNUJP1wg9d/U1U2VXWVfgZH2Bi81FTnaXuimuVVmc6WRxprP+eYFG6BCqIylYR7sQz9eEIB0x/grR/joCTVLcCwH43vX6f5Y37ndL6KpuqCyA0lQoObIdzvRcY1+43bMdZQn1fM36BeL7NFZqzUgB3wJVWR1kldaSWVZDZkkNaUVV7CusJKusFpf7+Go9JthMUaWNC7vF0L9tGMF+3lnWwWLUERfSzJbFEOJMqSpGaykBh3YRWLqNwNIdBJZux2gvP+ZhTmMg5fHDOZx4IYcTRuGqKobv/g69roMALw+XN5ghINK7xxRCiD9IL/f+7NABJs9a8gBBJoW/tDPyl3aezx9Wp8qOEhfbil3sK3Oz95CLjMNuqh2wo8TNjpIT57EYIMysEG45svkphJkVAk1HNqNCoBECTQoBRgV/I+hPo0f/91mFEBpRdEd62NtA8gWefQ4rHEqHkn1QlgGHMjxFvrXcc3/tmXVANnctpoB/4403+O9//0thYSG9e/fmtddeY+DAgVrH8ipVVSmvdVBYaaWofrNRWGmluNJKYaWVg+VWymrsf3qMUH8j3eOD6ZMYSp/EMPokhlJUaeXS19YwqnM07SMDmvAnEqL5UVw2zNV5WKpzsFTl4FeRjn/5fvzK044r1gHcejOVUf2pjB1CRey5VEf0+MMEK8WeLwHRnmFdQgjhIyxH/pTd93MjDMUFll8fcMLC2GJQGBhnYGDcb39LHS6VzAo3WZVuso98zapwc7BGpbjGU9xbnXDQqXKw5iRDC8/QlN5GEgL1+BvxFP8GhQAj+B85EeBnULDoFSwGMOuR6/eFaApGC8T08GxHOWohaw2sfRUSBmiXrRG1iAL+s88+Y9q0abz11lsMGjSImTNnMmbMGFJTU4mObhkTRKUXVzH21TXYnad3FjzU30h8iB/xoRYSw/1pFxFAuwh/wgNMx7ypFFVa5Xp10bKpKjqXFb2jGoO9AmNdKUZrKca6EozWUkx1pRitJfX7TbVFKJz4w5+KjrqABKpDO1Md2oXqkC7UBrdH1f1u1MrhrGOfJNeqCyF8VFygjhmjLFidp35sQ+RXu3ljq53txW5O0udwQvEBOuIDdAyJP3b/7jIX/1hp5cpOBsx6hSq7Wr9ZnVDnVKlzenr365xgb+Cggne2OwDHKR93lFnvOQFytKi3GBQsejAf+WoxKDjdKnpFwaQHs97z9ehto85zDIPOc9uoA6NewXD09pHvj94++rhQC3QK08sJBNF6Gf1/W8oupI22WRpJiyjgZ8yYwW233cbkyZMBeOutt1i4cCEffPABDz30kMbpvCPc6Kov3o04sODAjB0Ldsw4sCi2+n3+2DBaXWAFiqAIlSLg1z85tgJcooPk4oOEVZv+cO8pzmKf7AL6Uzz3z4qk0zv2qZzkuac47slzNeK/h0Y/78mee3ZtdBavqwKqC53bgVK/OY/7XnE7jtmnd9Sid1ajc9Sid9agd1Sjd9aiqA37pOZSdVgxYsWEVTVRi5la1YwVE+5KnWfG1Jx8YNnpH1SuVRdC+KC4QO/PZdOYPfvD2hhOK7PTrVLrgBqHSq1TxeYEqwtsThWry1PoW51QVONmZZ6L0UkGzHqocUCtQ6XG+bvnOzwnBVy/e1uzuTxbRf17nfdHBZyM50QAmHSeEwPG39026ZVj7tfrwKADveI5EeD5ime/Avoj+377/tjH6RTPpuAZeaBw9LYni3L0viO3jz72mPuOeYznGPzheb8/pjiW/LP8QZUZXOfQrdxJUvypH+5rfL6At9vtbN68mYcffrh+n06nY/To0axbt+6Ez7HZbNhsv61pXlHhmXSqsvIMZ4NuAvrKfBapdxOlVGBWTuNU+Jm8T6w9g+cI0cy4j2wn+i2xK37UGQKxGYKxGoKwGYKw6oKwHvneandSe3APtuj+4B/uvVB6I+RXAs33b4wQQjSlKUk66lzeLTv89CqHCtwcOsPnm49swcfs0+O2BRDrqqWDv9PzgD/hUsHpBodbwaGC68hXh1vB4QanW/HcryoU1ulZX2KiY5ADf4OKyw1OVcF15HlO1XM8l+o5lX70tksF95H73KpSf1v9QwlnPbIJ0ToZgdv5594yJrRp3p+9jtafagM68ny+gC8tLcXlchETE3PM/piYGPbt23fC5zz77LM88cQTx+1PTExslIxCiOaiivpr0k9qaWMHEUII4UOebqTjykVWQjSeO2fCnVqHOE1VVVWEhISc1mN9voA/Ew8//DDTpk2r/97tdnPo0CEiIiKa5TVDlZWVJCYmkpubS3Bw8KmfIJoFaTffJO3mu6TtfJO0m2+SdvNN0m6+S9rON52q3VRVpaqqivj40x/r7/MFfGRkJHq9nqKiomP2FxUVERsbe8LnmM1mzOZjx0CFhoY2VkSvCQ4Oll9YHyTt5puk3XyXtJ1vknbzTdJuvknazXdJ2/mmk7Xb6fa8H+X9mUmamMlkon///ixb9ttkUm63m2XLljFkyBANkwkhhBBCCCGEEN7j8z3wANOmTWPSpEkMGDCAgQMHMnPmTGpqaupnpRdCCCGEEEIIIXxdiyjgx48fT0lJCf/+978pLCykT58+LF68+LiJ7XyV2WzmscceO27Yv2jepN18k7Sb75K2803Sbr5J2s03Sbv5Lmk739QY7aaoDZmzXgghhBBCCCGEEJrw+WvghRBCCCGEEEKI1kAKeCGEEEIIIYQQwgdIAS+EEEIIIYQQQvgAKeCFEEIIIYQQQggfIAV8M/HGG2/Qrl07LBYLgwYNYsOGDX/62N27d3P11VfTrl07FEVh5syZTRdUHKMh7fbuu+8ybNgwwsLCCAsLY/To0Sd9vGg8DWm3+fPnM2DAAEJDQwkICKBPnz58/PHHTZhW/F5D2u735s2bh6IoXHHFFY0bUJxQQ9ptzpw5KIpyzGaxWJowrTiqob9v5eXlTJ06lbi4OMxmMykpKSxatKiJ0oqjGtJuI0eOPO73TVEULrnkkiZMLKDhv28zZ86kc+fO+Pn5kZiYyP3334/Vam2itOL3GtJ2DoeDJ598kuTkZCwWC71792bx4sUNe0FVaG7evHmqyWRSP/jgA3X37t3qbbfdpoaGhqpFRUUnfPyGDRvU6dOnq59++qkaGxurvvzyy00bWKiq2vB2mzBhgvrGG2+oW7duVffu3avefPPNakhIiJqXl9fEyVu3hrbb8uXL1fnz56t79uxR09PT1ZkzZ6p6vV5dvHhxEycXDW27ozIzM9WEhAR12LBh6uWXX940YUW9hrbb7Nmz1eDgYPXgwYP1W2FhYROnFg1tN5vNpg4YMEAdO3asumbNGjUzM1NdsWKFum3btiZO3ro1tN3KysqO+V3btWuXqtfr1dmzZzdt8Fauoe02d+5c1Ww2q3PnzlUzMzPVH3/8UY2Li1Pvv//+Jk4uGtp2Dz74oBofH68uXLhQzcjIUGfNmqVaLBZ1y5Ytp/2aUsA3AwMHDlSnTp1a/73L5VLj4+PVZ5999pTPTUpKkgJeI2fTbqqqqk6nUw0KClI//PDDxoooTuBs201VVbVv377qo48+2hjxxEmcSds5nU516NCh6nvvvadOmjRJCngNNLTdZs+erYaEhDRROvFnGtpub775ptqhQwfVbrc3VURxAmf7Hvfyyy+rQUFBanV1dWNFFCfQ0HabOnWqev755x+zb9q0aeq5557bqDnF8RradnFxcerrr79+zL6rrrpKnThx4mm/pgyh15jdbmfz5s2MHj26fp9Op2P06NGsW7dOw2TiZLzRbrW1tTgcDsLDwxsrpviDs203VVVZtmwZqampDB8+vDGjij8407Z78skniY6O5pZbbmmKmOIPzrTdqqurSUpKIjExkcsvv5zdu3c3RVxxxJm027fffsuQIUOYOnUqMTEx9OjRg2eeeQaXy9VUsVs9b3w2ef/997n++usJCAhorJjiD86k3YYOHcrmzZvrh2ofOHCARYsWMXbs2CbJLDzOpO1sNttxl4X5+fmxZs2a035dw5nFFd5SWlqKy+UiJibmmP0xMTHs27dPo1TiVLzRbv/4xz+Ij48/5pdeNK4zbbeKigoSEhKw2Wzo9XpmzZrFhRde2Nhxxe+cSdutWbOG999/n23btjVBQnEiZ9JunTt35oMPPqBXr15UVFTw4osvMnToUHbv3k2bNm2aInardybtduDAAX7++WcmTpzIokWLSE9P56677sLhcPDYY481RexW72w/m2zYsIFdu3bx/vvvN1ZEcQJn0m4TJkygtLSU8847D1VVcTqd3HHHHfzzn/9sisjiiDNpuzFjxjBjxgyGDx9OcnIyy5YtY/78+Q062Sk98EJo4LnnnmPevHksWLBAJmfyAUFBQWzbto2NGzfy9NNPM23aNFasWKF1LHESVVVV3Hjjjbz77rtERkZqHUc0wJAhQ7jpppvo06cPI0aMYP78+URFRfH2229rHU2chNvtJjo6mnfeeYf+/fszfvx4HnnkEd566y2to4nT9P7779OzZ08GDhyodRRxCitWrOCZZ55h1qxZbNmyhfnz57Nw4UL+85//aB1NnMIrr7xCp06d6NKlCyaTibvvvpvJkyej051+WS498BqLjIxEr9dTVFR0zP6ioiJiY2M1SiVO5Wza7cUXX+S5555j6dKl9OrVqzFjij8403bT6XR07NgRgD59+rB3716effZZRo4c2Zhxxe80tO0yMjLIyspi3Lhx9fvcbjcABoOB1NRUkpOTGze08Mp7nNFopG/fvqSnpzdGRHECZ9JucXFxGI1G9Hp9/b6uXbtSWFiI3W7HZDI1amZxdr9vNTU1zJs3jyeffLIxI4oTOJN2+9e//sWNN97IrbfeCkDPnj2pqalhypQpPPLIIw0qBsWZO5O2i4qK4uuvv8ZqtVJWVkZ8fDwPPfQQHTp0OO3XldbVmMlkon///ixbtqx+n9vtZtmyZQwZMkTDZOJkzrTdXnjhBf7zn/+wePFiBgwY0BRRxe946/fN7XZjs9kaI6L4Ew1tuy5durBz5062bdtWv1122WWMGjWKbdu2kZiY2JTxWy1v/M65XC527txJXFxcY8UUf3Am7XbuueeSnp5ef6IMIC0tjbi4OCnem8jZ/L598cUX2Gw2/vrXvzZ2TPEHZ9JutbW1xxXpR0+eqaraeGHFMc7md85isZCQkIDT6eSrr77i8ssvP/0XPoPJ9oSXzZs3TzWbzeqcOXPUPXv2qFOmTFFDQ0Prl8258cYb1Yceeqj+8TabTd26dau6detWNS4uTp0+fbq6detWdf/+/Vr9CK1SQ9vtueeeU00mk/rll18es2RLVVWVVj9Cq9TQdnvmmWfUJUuWqBkZGeqePXvUF198UTUYDOq7776r1Y/QajW07f5IZqHXRkPb7YknnlB//PFHNSMjQ928ebN6/fXXqxaLRd29e7dWP0Kr1NB2y8nJUYOCgtS7775bTU1NVb///ns1Ojpafeqpp7T6EVqlM/07ed5556njx49v6rjiiIa222OPPaYGBQWpn376qXrgwAF1yZIlanJysnrddddp9SO0Wg1tu/Xr16tfffWVmpGRoa5atUo9//zz1fbt26uHDx8+7deUAr6ZeO2119S2bduqJpNJHThwoLp+/fr6+0aMGKFOmjSp/vvMzEwVOG4bMWJE0wdv5RrSbklJSSdst8cee6zpg7dyDWm3Rx55RO3YsaNqsVjUsLAwdciQIeq8efM0SC1UtWFt90dSwGunIe1233331T82JiZGHTt2bIPWxxXe09Dft7Vr16qDBg1SzWaz2qFDB/Xpp59WnU5nE6cWDW23ffv2qYC6ZMmSJk4qfq8h7eZwONTHH39cTU5OVi0Wi5qYmKjeddddDSoChfc0pO1WrFihdu3aVTWbzWpERIR64403qvn5+Q16PUVVZZyFEEIIIYQQQgjR3Mk18EIIIYQQQgghhA+QAl4IIYQQQgghhPABUsALIYQQQgghhBA+QAp4IYQQQgghhBDCB0gBL4QQQgghhBBC+AAp4IUQQgghhBBCCB8gBbwQQgghhBBCCOEDpIAXQgghhBBCCCF8gBTwQgghhI+ZM2cOoaGhDXrOzTffzBVXXNEoeZqTdu3aMXPmTK1jCCGEEI1CCnghhBCimfizInvFihUoikJ5eTkA48ePJy0trWnDncIfM57qcUe3qKgoxo4dy86dOxv0en92EmPjxo1MmTKlQccSQgghfIUU8EIIIYSP8fPzIzo6WusYZyU1NZWDBw/y448/YrPZuOSSS7Db7Wd93KioKPz9/b2QUAghhGh+pIAXQgghfMyJep+feuopoqOjCQoK4tZbb+Whhx6iT58+xz33xRdfJC4ujoiICKZOnYrD4ai/z2azMX36dBISEggICGDQoEGsWLGi/v7s7GzGjRtHWFgYAQEBdO/enUWLFpGVlcWoUaMACAsLQ1EUbr755pP+DNHR0cTGxtKvXz/uu+8+cnNz2bdvX/39M2bMoGfPngQEBJCYmMhdd91FdXU14OnFnzx5MhUVFfU9+Y8//jhw/BD6nJwcLr/8cgIDAwkODua6666jqKjo1P/IQgghRDMkBbwQQgjh4+bOncvTTz/N888/z+bNm2nbti1vvvnmcY9bvnw5GRkZLF++nA8//JA5c+YwZ86c+vvvvvtu1q1bx7x589ixYwfXXnstF110Efv37wdg6tSp2Gw2Vq1axc6dO3n++ecJDAwkMTGRr776CvitZ/2VV145rewVFRXMmzcPAJPJVL9fp9Px6quvsnv3bj788EN+/vlnHnzwQQCGDh3KzJkzCQ4O5uDBgxw8eJDp06cfd2y3283ll1/OoUOHWLlyJT/99BMHDhxg/Pjxp/cPK4QQQjQzBq0DCCGEEOI333//PYGBgcfsc7lcJ33Oa6+9xi233MLkyZMB+Pe//82SJUvqe6yPCgsL4/XXX0ev19OlSxcuueQSli1bxm233UZOTg6zZ88mJyeH+Ph4AKZPn87ixYuZPXs2zzzzDDk5OVx99dX07NkTgA4dOtQfOzw8HPD0rJ/OBHtt2rQBoKamBoDLLruMLl261N9/33331d9u164dTz31FHfccQezZs3CZDIREhKCoijExsb+6WssW7aMnTt3kpmZSWJiIgAfffQR3bt3Z+PGjZxzzjmnzCmEEEI0J9IDL4QQQjQjo0aNYtu2bcds77333kmfk5qaysCBA4/Z98fvAbp3745er6//Pi4ujuLiYgB27tyJy+UiJSWFwMDA+m3lypVkZGQAcO+99/LUU09x7rnn8thjj7Fjx44z/jlXr17N5s2bmTNnDikpKbz11lvH3L906VIuuOACEhISCAoK4sYbb6SsrIza2trTfo29e/eSmJhYX7wDdOvWjdDQUPbu3XvG2YUQQgitSA+8EEII0YwEBATQsWPHY/bl5eV55dhGo/GY7xVFwe12A1BdXY1er2fz5s3HFPlA/YiAW2+9lTFjxrBw4UKWLFnCs88+y0svvcQ999zT4Czt27cnNDSUzp07U1xczPjx41m1ahUAWVlZXHrppdx55508/fTThIeHs2bNGm655RbsdrtMUieEEKLVkh54IYQQwsd17tyZjRs3HrPvj9+fSt++fXG5XBQXF9OxY8djtt8PU09MTOSOO+5g/vz5PPDAA7z77rvAb9evn2q4/4lMnTqVXbt2sWDBAgA2b96M2+3mpZdeYvDgwaSkpFBQUHDMc0wm0ylfq2vXruTm5pKbm1u/b8+ePZSXl9OtW7cG5xRCCCG0JgW8EEII4ePuuece3n//fT788EP279/PU089xY4dO1AU5bSPkZKSwsSJE7npppuYP38+mZmZbNiwgWeffZaFCxcCnuvSf/zxRzIzM9myZQvLly+na9euACQlJaEoCt9//z0lJSXHXX9/Mv7+/tx222089thjqKpKx44dcTgcvPbaaxw4cICPP/74uCH27dq1o7q6mmXLllFaWnrCofWjR4+mZ8+eTJw4kS1btrBhwwZuuukmRowYwYABA047nxBCCNFcSAEvhBBC+LiJEyfy8MMPM336dPr160dmZiY333wzFoulQceZPXs2N910Ew888ACdO3fmiiuuYOPGjbRt2xbw9K5PnTqVrl27ctFFF5GSksKsWbMASEhI4IknnuChhx4iJiaGu+++u0Gvfffdd7N3716++OILevfuzYwZM3j++efp0aMHc+fO5dlnnz3m8UOHDuWOO+5g/PjxREVF8cILLxx3TEVR+OabbwgLC2P48OGMHj2aDh068NlnnzUomxBCCNFcKKqqqlqHEEIIIYR3XXjhhcTGxvLxxx9rHUUIIYQQXiKT2AkhhBA+rra2lrfeeosxY8ag1+v59NNPWbp0KT/99JPW0YQQQgjhRdIDL4QQQvi4uro6xo0bx9atW7FarXTu3JlHH32Uq666SutoQgghhPAiKeCFEEIIIYQQQggfIJPYCSGEEEIIIYQQPkAKeCGEEEIIIYQQwgdIAS+EEEIIIYQQQvgAKeCFEEIIIYQQQggfIAW8EEIIIYQQQgjhA6SAF0IIIYQQQgghfIAU8EIIIYQQQgghhA+QAl4IIYQQQgghhPAB/w/9EDtx+YLfgwAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA+4AAAK9CAYAAAC6iTZTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAAD5DklEQVR4nOzdd3hb5fnG8e+RvPeecWJn7wkJgUBiEkhCWQUKYUNZpUDLKKXsPX5lFkoLlFIoUEZpCy0jZEMWgWyyl53hxHEcr3jb0vn9IUvE2Iktx/aR7PtzXb60jo5uOwbr0fs+72uYpmkiIiIiIiIiIj7JZnUAERERERERETkyFe4iIiIiIiIiPkyFu4iIiIiIiIgPU+EuIiIiIiIi4sNUuIuIiIiIiIj4MBXuIiIiIiIiIj5MhbuIiIiIiIiID1PhLiIiIiIiIuLDVLiLiIiIiIiI+DAV7iIiYqmHHnoIwzA65bUmTZrEpEmTPLcXLFiAYRh89NFHnfL6V111FZmZmZ3yWm1VXl7OtddeS0pKCoZhcOutt1odSQ5jGAY333yz1TFERKSTqXAXEZF28+abb2IYhucrJCSEtLQ0pk6dyosvvsihQ4fa5XX27t3LQw89xOrVq9vlfO3Jl7O1xhNPPMGbb77JjTfeyNtvv83ll1/e4nMcDgdpaWkYhsEXX3zRCSn9z/r167nssstIT08nODiYtLQ0Lr30UtavX9/k2CVLlvDQQw9RUlLS+UFFRMQnqXAXEZF298gjj/D222/z5z//mVtuuQWAW2+9lWHDhrF27dpGx953331UVVV5df69e/fy8MMPe10cz5o1i1mzZnn1HG8dLdtf/vIXNm/e3KGvf6zmzZvHCSecwIMPPshll13GmDFjWvWcffv2kZmZybvvvtsJKf3Lv//9b0aPHs3cuXO5+uqr+dOf/sQ111zD/PnzGT16NP/5z38aHb9kyRIefvhhFe4iIuIRYHUAERHpeqZPn85xxx3nuX333Xczb948zjzzTM4++2w2btxIaGgoAAEBAQQEdOyfo8rKSsLCwggKCurQ12lJYGCgpa/fGgUFBQwePNir57zzzjuMHj2aK6+8knvuuYeKigrCw8M7KGHzrHjN1ti+fTuXX345vXv35uuvvyYxMdHz2K9//WtOPvlkLr/8ctauXUvv3r0tTPqD6upqgoKCsNk0viMi4iv0f2QREekUp556Kvfffz87d+7knXfe8dzfXI/77NmzmTBhAjExMURERDBgwADuuecewNWXfvzxxwNw9dVXe6blv/nmm4Crj33o0KGsWLGCU045hbCwMM9zf9zj7uZwOLjnnntISUkhPDycs88+m927dzc6JjMzk6uuuqrJcw8/Z0vZmutxr6io4I477iAjI4Pg4GAGDBjAM888g2majY5z9zZ//PHHDB06lODgYIYMGcLMmTOb/4H/SEFBAddccw3JycmEhIQwYsQI3nrrLc/j7n7/nJwcPvvsM0/23Nzco563qqqK//znP8yYMYMLL7yQqqoqPvnkE8/jzzzzDIZhsHPnzibPvfvuuwkKCqK4uNhz37Jly5g2bRrR0dGEhYUxceJEFi9e3Oh57t+ZDRs2cMkllxAbG8uECRMAWLt2LVdddRW9e/cmJCSElJQUfv7zn3Pw4MEmr79gwQKOO+44QkJC6NOnD6+++uoR11x45513GDNmDKGhocTFxTFjxowmvyPNefrpp6msrOS1115rVLQDJCQk8Oqrr1JRUcHvf/97z/d25513ApCVlXXEf4fW/B7k5eXx85//nOTkZM9xb7zxRpOfgWEYvP/++9x3332kp6cTFhZGWVkZdXV1PPzww/Tr14+QkBDi4+OZMGECs2fPbvH7FhGR9qURdxER6TSXX34599xzD7NmzeK6665r9pj169dz5plnMnz4cB555BGCg4PZtm2bp3gbNGgQjzzyCA888ADXX389J598MgAnnnii5xwHDx5k+vTpzJgxg8suu4zk5OSj5nr88ccxDIO77rqLgoICXnjhBaZMmcLq1as9MwNaozXZDmeaJmeffTbz58/nmmuuYeTIkXz55Zfceeed5OXl8fzzzzc6ftGiRfz73//ml7/8JZGRkbz44oucf/757Nq1i/j4+CPmqqqqYtKkSWzbto2bb76ZrKws/vnPf3LVVVdRUlLCr3/9awYNGsTbb7/NbbfdRo8ePbjjjjsAmhSbP/bf//6X8vJyZsyYQUpKCpMmTeLdd9/lkksuAeDCCy/kt7/9LR9++KGnIHX78MMPOf3004mNjQVcU+6nT5/OmDFjePDBB7HZbPztb3/j1FNPZeHChYwdO7bR83/2s5/Rr18/nnjiCc8HHbNnz2bHjh1cffXVpKSksH79el577TXWr1/PN9984ynKV61axbRp00hNTeXhhx/G4XDwyCOPNPv9Pv7449x///1ceOGFXHvttRw4cICXXnqJU045hVWrVhETE3PEn8///vc/MjMzPb8LP3bKKaeQmZnJZ599BsB5553Hli1beO+993j++edJSEho8u/Qmt+D/fv3c8IJJ3g+8ElMTOSLL77gmmuuoaysrMmig48++ihBQUH85je/oaamhqCgIB566CGefPJJrr32WsaOHUtZWRnLly9n5cqVnHbaaUf8nkVEpAOYIiIi7eRvf/ubCZjffffdEY+Jjo42R40a5bn94IMPmof/OXr++edNwDxw4MARz/Hdd9+ZgPm3v/2tyWMTJ040AfOVV15p9rGJEyd6bs+fP98EzPT0dLOsrMxz/4cffmgC5h/+8AfPfb169TKvvPLKFs95tGxXXnml2atXL8/tjz/+2ATMxx57rNFxF1xwgWkYhrlt2zbPfYAZFBTU6L41a9aYgPnSSy81ea3DvfDCCyZgvvPOO577amtrzfHjx5sRERGNvvdevXqZP/nJT456vsOdeeaZ5kknneS5/dprr5kBAQFmQUGB577x48ebY8aMafS8b7/91gTMv//976ZpmqbT6TT79etnTp061XQ6nZ7jKisrzaysLPO0007z3Of+nbn44oub5KmsrGxy33vvvWcC5tdff+2576yzzjLDwsLMvLw8z31bt241AwICGv0+5ubmmna73Xz88ccbnfP77783AwICmtx/uJKSEhMwzznnnCMeY5qmefbZZ5uA59/h6aefNgEzJyenybGt/T245pprzNTUVLOwsLDR82fMmGFGR0d7fk7u/wZ69+7d5Gc3YsQIr34XRESk42iqvIiIdKqIiIijri7vHr385JNPcDqdbXqN4OBgrr766lYff8UVVxAZGem5fcEFF5Camsrnn3/eptdvrc8//xy73c6vfvWrRvffcccdmKbZZIX2KVOm0KdPH8/t4cOHExUVxY4dO1p8nZSUFC6++GLPfYGBgfzqV7+ivLycr776qk35Dx48yJdfftnovOeffz6GYfDhhx967rvoootYsWIF27dv99z3wQcfEBwczDnnnAPA6tWr2bp1K5dccgkHDx6ksLCQwsJCKioqmDx5Ml9//XWT34df/OIXTTIdPkOiurqawsJCTjjhBABWrlwJuFoj5syZw7nnnktaWprn+L59+zJ9+vRG5/v3v/+N0+nkwgsv9GQqLCwkJSWFfv36MX/+/CP+fNy/54f/bjXH/XhZWdlRj3Nr6ffANE3+9a9/cdZZZ2GaZqPcU6dOpbS01POzcLvyyiubzC6JiYlh/fr1bN26tVW5RESk46hwFxGRTlVeXn7UQuaiiy7ipJNO4tprryU5OZkZM2bw4YcfelXEp6ene7UQXb9+/RrdNgyDvn37ttjffax27txJWlpak5/HoEGDPI8frmfPnk3OERsb26hH/Eiv069fvyaLjR3pdVrrgw8+oK6ujlGjRrFt2za2bdtGUVER48aNa7S6/M9+9jNsNhsffPAB4Cos//nPfzJ9+nSioqIAPMXhlVdeSWJiYqOv119/nZqaGkpLSxu9flZWVpNMRUVF/PrXvyY5OZnQ0FASExM9x7mfX1BQQFVVFX379m3y/B/ft3XrVkzTpF+/fk1ybdy4kYKCgiP+fNz/ri1tg9jaAt+tpd+DAwcOUFJS4umrP/zL/YHWj3M397N85JFHKCkpoX///gwbNow777yzya4QIiLSOdTjLiIinWbPnj2UlpY2WzC5hYaG8vXXXzN//nw+++wzZs6cyQcffMCpp57KrFmzsNvtLb6ON33prdXcgmXgGr1tTab2cKTXMX+0kF1ncRfnJ510UrOP79ixg969e5OWlsbJJ5/Mhx9+yD333MM333zDrl27+L//+z/Pse4PZp5++mlGjhzZ7PkiIiIa3W7u3/nCCy9kyZIl3HnnnYwcOZKIiAicTifTpk1r0wwOp9Pp2Z++uZ//jzMdLjo6mtTU1BaL3bVr15Kenu75EKMlLf0euL/Pyy67jCuvvLLZY4cPH97odnM/y1NOOYXt27fzySefMGvWLF5//XWef/55XnnlFa699tpWZRURkfahwl1ERDrN22+/DcDUqVOPepzNZmPy5MlMnjyZ5557jieeeIJ7772X+fPnM2XKlCMW0W3146nApmmybdu2RsVNbGxss/tq79y5s9E2Xt5k69WrF3PmzOHQoUONRls3bdrkebw99OrVi7Vr1+J0OhuNuh/L6+Tk5LBkyRJuvvlmJk6c2Ogxp9PJ5Zdfzj/+8Q/uu+8+wDWT4pe//CWbN2/mgw8+ICwsjLPOOsvzHPfU76ioKKZMmeJ1HoDi4mLmzp3Lww8/zAMPPOC5/8f/vklJSYSEhLBt27Ym5/jxfX369ME0TbKysujfv7/Xmc4880z+8pe/sGjRIs/K94dbuHAhubm53HDDDZ77jvX3OzExkcjISBwOR5t/lm5xcXFcffXVXH311ZSXl3PKKafw0EMPqXAXEelkmiovIiKdYt68eTz66KNkZWVx6aWXHvG4oqKiJve5R2BramoAPPt1N1dIt8Xf//73RtOZP/roI/bt29eo37lPnz5888031NbWeu779NNPm2wJ5k22M844A4fDwR//+MdG9z///PMYhtGk37qtzjjjDPLz8z1T1QHq6+t56aWXiIiIaFJ4t4Z7tP23v/0tF1xwQaOvCy+8kIkTJzaaLn/++edjt9t57733+Oc//8mZZ57ZaN/1MWPG0KdPH5555hnKy8ubvN6BAwdazOQeif7xDIQXXnihyXFTpkzh448/Zu/evZ77t23b1mRdgfPOOw+73c7DDz/c5LymaTa7zdzh7rzzTkJDQ7nhhhuaHFtUVMQvfvELwsLCGq24f6y/33a7nfPPP59//etfrFu3rsnjrflZAk3yRkRE0LdvX89/hyIi0nk04i4iIu3uiy++YNOmTdTX17N//37mzZvH7Nmz6dWrF//9738JCQk54nMfeeQRvv76a37yk5/Qq1cvCgoK+NOf/kSPHj08I5Z9+vQhJiaGV155hcjISMLDwxk3blyzfbqtERcXx4QJE7j66qvZv38/L7zwAn379m20Zd21117LRx99xLRp07jwwgvZvn0777zzTqNFwrzNdtZZZ5Gdnc29995Lbm4uI0aMYNasWXzyySfceuutTc7dVtdffz2vvvoqV111FStWrCAzM5OPPvqIxYsX88ILL7S6t/pw7777LiNHjiQjI6PZx88++2xuueUWVq5cyejRo0lKSiI7O5vnnnuOQ4cOcdFFFzU63maz8frrrzN9+nSGDBnC1VdfTXp6Onl5ecyfP5+oqCj+97//HTVTVFQUp5xyCr///e+pq6sjPT2dWbNmkZOT0+TYhx56iFmzZnHSSSdx4403ej5AGTp0KKtXr/Yc16dPHx577DHuvvtucnNzOffcc4mMjCQnJ4f//Oc/XH/99fzmN785YqZ+/frx1ltvcemllzJs2DCuueYasrKyyM3N5a9//SuFhYW89957jf6tx4wZA8C9997LjBkzCAwM5Kyzzmr0QUdLnnrqKebPn8+4ceO47rrrGDx4MEVFRaxcuZI5c+Y0+wHZjw0ePJhJkyYxZswY4uLiWL58OR999BE333xzq3OIiEg7sWYxexER6Yrc28G5v4KCgsyUlBTztNNOM//whz802nbM7cfbwc2dO9c855xzzLS0NDMoKMhMS0szL774YnPLli2NnvfJJ5+YgwcP9mzf5d5+beLEieaQIUOazXek7eDee+898+677zaTkpLM0NBQ8yc/+Ym5c+fOJs9/9tlnzfT0dDM4ONg86aSTzOXLlzc559Gy/Xg7ONM0zUOHDpm33XabmZaWZgYGBpr9+vUzn3766UZbopmmaxuwm266qUmmI21T92P79+83r776ajMhIcEMCgoyhw0b1uyWda3ZDm7FihUmYN5///1HPCY3N9cEzNtuu81z31/+8hcTMCMjI82qqqpmn7dq1SrzvPPOM+Pj483g4GCzV69e5oUXXmjOnTvXc4z7d6a5LQP37Nlj/vSnPzVjYmLM6Oho82c/+5m5d+9eEzAffPDBRsfOnTvXHDVqlBkUFGT26dPHfP3118077rjDDAkJaXLef/3rX+aECRPM8PBwMzw83Bw4cKB50003mZs3bz7qz8pt7dq15sUXX2ympqaagYGBZkpKinnxxReb33//fbPHP/roo2Z6erpps9kabQ3nze/B/v37zZtuusnMyMjwvObkyZPN1157zXOM+7+Bf/7zn03O+dhjj5ljx441Y2JizNDQUHPgwIHm448/btbW1rbqexYRkfZjmKZFK9qIiIiI+Jhzzz1XW6CJiIjPUY+7iIiIdEtVVVWNbm/dupXPP/+cSZMmWRNIRETkCDTiLiIiIt1SamoqV111Fb1792bnzp38+c9/pqamhlWrVtGvXz+r44mIiHhocToRERHplqZNm8Z7771Hfn4+wcHBjB8/nieeeEJFu4iI+ByNuIuIiIiIiIj4MPW4i4iIiIiIiPgwFe4iIiIiIiIiPkw97oDT6WTv3r1ERkZiGIbVcURERERERKSLM02TQ4cOkZaWhs129DF1Fe7A3r17ycjIsDqGiIiIiIiIdDO7d++mR48eRz1GhTsQGRkJuH5gUVFRFqcRERERERGRrq6srIyMjAxPPXo0KtzBMz0+KipKhbuIiIiIiIh0mta0a2txOhEREREREREfpsJdRERERERExIepcBcRERERERHxYepxFxERERERaYFpmtTX1+NwOKyOIn7CbrcTEBDQLluOq3AXERERERE5itraWvbt20dlZaXVUcTPhIWFkZqaSlBQ0DGdR4W7iIiIiIjIETidTnJycrDb7aSlpREUFNQuI6jStZmmSW1tLQcOHCAnJ4d+/fphs7W9U12Fu4iIiIiIyBHU1tbidDrJyMggLCzM6jjiR0JDQwkMDGTnzp3U1tYSEhLS5nNpcToREREREZEWHMtoqXRf7fV7o98+ERERERERER+mwl1ERERERETEh6lwFxEREREREa+8+eabxMTEHPN5DMPg448/PubzdHUq3EVERERERLqhq666inPPPdfqGNIKKtxFREREREREfJgKdxEREREREWnkueeeY9iwYYSHh5ORkcEvf/lLysvLmxz38ccf069fP0JCQpg6dSq7d+9u9Pgnn3zC6NGjCQkJoXfv3jz88MPU19d31rfRZahwFxERERERkUZsNhsvvvgi69ev56233mLevHn89re/bXRMZWUljz/+OH//+99ZvHgxJSUlzJgxw/P4woULueKKK/j1r3/Nhg0bePXVV3nzzTd5/PHHO/vb8Xsq3EVERERERKSRW2+9lezsbDIzMzn11FN57LHH+PDDDxsdU1dXxx//+EfGjx/PmDFjeOutt1iyZAnffvstAA8//DC/+93vuPLKK+nduzennXYajz76KK+++qoV35JfC7A6gIiIiIiIiPiWOXPm8OSTT7Jp0ybKysqor6+nurqayspKwsLCAAgICOD444/3PGfgwIHExMSwceNGxo4dy5o1a1i8eHGjEXaHw9HkPNIyFe4iIiIiIiLikZuby5lnnsmNN97I448/TlxcHIsWLeKaa66htra21QV3eXk5Dz/8MOedd16Tx0JCQto7dpemwl1EREREREQ8VqxYgdPp5Nlnn8Vmc3VX/3iaPEB9fT3Lly9n7NixAGzevJmSkhIGDRoEwOjRo9m8eTN9+/btvPBdlAp3ERERERGRbqq0tJTVq1c3ui8hIYG6ujpeeuklzjrrLBYvXswrr7zS5LmBgYHccsstvPjiiwQEBHDzzTdzwgkneAr5Bx54gDPPPJOePXtywQUXYLPZWLNmDevWreOxxx7rjG+vy9DidCIiIiIiIt3UggULGDVqVKOvt99+m+eee47/+7//Y+jQobz77rs8+eSTTZ4bFhbGXXfdxSWXXMJJJ51EREQEH3zwgefxqVOn8umnnzJr1iyOP/54TjjhBJ5//nl69erVmd9il2CYpmlaHcJqZWVlREdHU1paSlRUlNVxRERERETER1RXV5OTk0NWVpb6ssVrR/v98aYO1Yi7iIiIiIiIiA9T4S4iIiIiIiLiw1S4i4iIiIiIiPgwrSovIiIifiunsIKKmvp2O194cABZCeHtdj4REZH2oMJdRERE/FJOYQXZzyxo9/PO/80kFe8iIuJTVLiLiIiIX3KPtN+U3Zf0mNBjPl9eSRUvz9/WriP4IiIi7UGFu4iIiPi19JhQjZCLiEiXpsXpRERERERERHyYRtxFRERERES8lFdSRXFFbae9Xmx4ULu0BYl/UuEuIiIiIiLihbySKiY/u4DqOmenvWZIoI25d0zqtOI9NzeXrKwsVq1axciRI1v1nDfffJNbb72VkpISS3N0RSrcRUREREREvFBcUUt1nbPdFsdsiXvxzOKKWq9fb/fu3Tz44IPMnDmTwsJCUlNTOffcc3nggQeIj48/4vMyMjLYt28fCQkJrX6tiy66iDPOOMOrfNI6KtxFRERERETawNcXx9yxYwfjx4+nf//+vPfee2RlZbF+/XruvPNOvvjiC7755hvi4uKaPK+2tpagoCBSUlK8er3Q0FBCQzWdvyNocToREREREZEu6KabbiIoKIhZs2YxceJEevbsyfTp05kzZw55eXnce++9AGRmZvLoo49yxRVXEBUVxfXXX09ubi6GYbB69WrP+f773//Sr18/QkJCyM7O5q233sIwDM/U+DfffJOYmBjP8Q899BAjR47k7bffJjMzk+joaGbMmMGhQ4c8x8ycOZMJEyYQExNDfHw8Z555Jtu3b++MH49fUeEuIiIiIiLSxRQVFfHll1/yy1/+sskoeEpKCpdeeikffPABpmkC8MwzzzBixAhWrVrF/fff3+R8OTk5XHDBBZx77rmsWbOGG264wVP4H8327dv5+OOP+fTTT/n000/56quveOqppzyPV1RUcPvtt7N8+XLmzp2LzWbjpz/9KU5n560f4A80VV5ERERERKSL2bp1K6ZpMmjQoGYfHzRoEMXFxRw4cACAU089lTvuuMPzeG5ubqPjX331VQYMGMDTTz8NwIABA1i3bh2PP/74UXM4nU7efPNNIiMjAbj88suZO3eu53nnn39+o+PfeOMNEhMT2bBhA0OHDm39N9zFacRdRERERESki3KPqLfkuOOOO+rjmzdv5vjjj29039ixY1s8b2ZmpqdoB0hNTaWgoMBze+vWrVx88cX07t2bqKgoMjMzAdi1a1ercncXKtxFRERERES6mL59+2IYBhs3bmz28Y0bNxIbG0tiYiIA4eEds8heYGBgo9uGYTSaBn/WWWdRVFTEX/7yF5YtW8ayZcsA1wJ58gMV7iIiIiIiIl1MfHw8p512Gn/605+oqqpq9Fh+fj7vvvsuF110EYZhtOp8AwYMYPny5Y3u++67744p48GDB9m8eTP33XcfkydP9kzfl6bU4y4iIiIiItIGeSVVLR9k4ev88Y9/5MQTT2Tq1Kk89thjjbaDS09Pb7E//XA33HADzz33HHfddRfXXHMNq1ev5s033wRodfH/Y7GxscTHx/Paa6+RmprKrl27+N3vftemc3V1KtxFRERERES8EBseREigjZfnb+u01wwJtBEbHuTVc/r168fy5ct58MEHufDCCykqKiIlJYVzzz2XBx98sNk93I8kKyuLjz76iDvuuIM//OEPjB8/nnvvvZcbb7yR4OBgb78dAGw2G++//z6/+tWvGDp0KAMGDODFF19k0qRJbTpfV2aYrV2toAsrKysjOjqa0tJSoqKirI4jIiIirbAur5QzX1rEEz8dRlbCsfdm5hRWcM9/vufTWyYwND26HRKKSFdQXV1NTk4OWVlZhISEeO7PK6miuKLz+rBjw4NIjwlt+cBO9Pjjj/PKK6+we/duq6P4rCP9/oB3dahG3EVERERERLyUHhPqc4V0R/vTn/7E8ccfT3x8PIsXL+bpp5/m5ptvtjpWt6DCXURERERERFq0detWHnvsMYqKiujZsyd33HEHd999t9WxugUV7iIiIiIiItKi559/nueff97qGN2StoMTERERERER8WEq3EVERERERER8mAp3ERERERERER+mwl1ERERERETEh6lwFxEREREREfFhWlVeRERERETEWyW7ofJg571eWDzEZHTe64lPUeEuIiIiIiLijZLd8PLxUFfVea8ZGAo3fafi/RhkZmZy6623cuutt7bbOSdNmsTIkSN54YUX2u2czVHhLiIiIiIi4o3Kg66i/eQ7ILoTCunS3bDwWdfrtrJwf+WVV7jzzjspLi4mIMBV9pWXlxMbG8tJJ53EggULPMcuWLCA7Oxstm3bRp8+fY54TvdxxcXFxMTEHMt3JF5S4S4iIiIiItIW0RkQ39fqFM3Kzs6mvLyc5cuXc8IJJwCwcOFCUlJSWLZsGdXV1YSEhAAwf/58evbsedSivT2ZponD4fB8oNDRamtrCQoK6pTX6ihanE5ERERERKSLGTBgAKmpqU1G1s855xyysrL45ptvGt2fnZ3N22+/zXHHHUdkZCQpKSlccsklFBQUAJCbm0t2djYAsbGxGIbBVVddBYDT6eTJJ58kKyuL0NBQRowYwUcffdTo/IZh8MUXXzBmzBiCg4NZtGgRkyZN4pZbbuHWW28lNjaW5ORk/vKXv1BRUcHVV19NZGQkffv25YsvvvCcy+FwcM0113hea8CAAfzhD39o9L1fddVVnHvuuTz++OOkpaUxYMCAZn9Gr7/+OjExMcydOxeAdevWMX36dCIiIkhOTubyyy+nsLDQc3xFRQVXXHEFERERpKam8uyzz7bhX6ZtVLiLiIiIiIh0QdnZ2cyfP99ze/78+UyaNImJEyd67q+qqmLZsmVkZ2dTV1fHo48+ypo1a/j444/Jzc31FOcZGRn861//AmDz5s3s27fPUzA/+eST/P3vf+eVV15h/fr13HbbbVx22WV89dVXjfL87ne/46mnnmLjxo0MHz4cgLfeeouEhAS+/fZbbrnlFm688UZ+9rOfceKJJ7Jy5UpOP/10Lr/8ciorKwHXhwQ9evTgn//8Jxs2bOCBBx7gnnvu4cMPP2z0WnPnzmXz5s3Mnj2bTz/9tMnP5ve//z2/+93vmDVrFpMnT6akpIRTTz2VUaNGsXz5cmbOnMn+/fu58MILPc+58847+eqrr/jkk0+YNWsWCxYsYOXKlcfyT9RqmiovIiIiIiLSBWVnZ3PrrbdSX19PVVUVq1atYuLEidTV1fHKK68AsHTpUmpqasjOzqZnz56e5/bu3ZsXX3yR448/nvLyciIiIoiLiwMgKSnJ0+NeU1PDE088wZw5cxg/frznuYsWLeLVV19l4sSJnnM+8sgjnHbaaY0yjhgxgvvuuw+Au+++m6eeeoqEhASuu+46AB544AH+/Oc/s3btWk444QQCAwN5+OGHPc/Pyspi6dKlfPjhh42K7PDwcF5//fVmp8jfddddvP3223z11VcMGTIEgD/+8Y+MGjWKJ554wnPcG2+8QUZGBlu2bCEtLY2//vWvvPPOO0yePBlwfejQo0cPb/5J2kyFu4iIiIiISBc0adIkKioq+O677yguLqZ///4kJiYyceJErr76aqqrq1mwYAG9e/emZ8+erFixgoceeog1a9ZQXFyM0+kEYNeuXQwePLjZ19i2bRuVlZVNCvLa2lpGjRrV6L7jjjuuyfPdI+8Adrud+Ph4hg0b5rkvOTkZwDNlH+Dll1/mjTfeYNeuXVRVVVFbW8vIkSMbnXfYsGHNFu3PPvssFRUVLF++nN69e3vuX7NmDfPnzyciIqLJc7Zv3+55nXHjxnnuj4uLO+I0/Pamwl1ERESkuzu4HWoOte85gyMhvnMWuhKR5vXt25cePXowf/58iouLPaPfaWlpZGRksGTJEubPn8+pp55KRUUFU6dOZerUqbz77rskJiaya9cupk6dSm1t7RFfo7y8HIDPPvuM9PT0Ro8FBwc3uh0eHt7k+YGBgY1uG4bR6D7DMAA8HyK8//77/OY3v+HZZ59l/PjxREZG8vTTT7Ns2bIWXwvg5JNP5rPPPuPDDz/kd7/7XaPv46yzzuL//u//mjwnNTWVbdu2NXu+zqLCXURERKQ7O7gdXhrdMee+ZaWKdxGLZWdns2DBAoqLi7nzzjs9959yyil88cUXfPvtt9x4441s2rSJgwcP8tRTT5GR4dpybvny5Y3O5R7BdjgcnvsGDx5McHAwu3btajQtvqMsXryYE088kV/+8pee+7Zv397q548dO5abb76ZadOmERAQwG9+8xsARo8ezb/+9S8yMzObXe2+T58+BAYGsmzZMk9LQXFxMVu2bOmU71uFu4iIiEh35h5pb8/9qN17Trf3KL6Irynd7fOvk52dzU033URdXV2jAnPixIncfPPN1NbWkp2dTUBAAEFBQbz00kv84he/YN26dTz66KONztWrVy8Mw+DTTz/ljDPOIDQ0lMjISH7zm99w22234XQ6mTBhAqWlpSxevJioqCiuvPLKNmdvTr9+/fj73//Ol19+SVZWFm+//TbfffcdWVlZrT7HiSeeyOeff8706dMJCAjg1ltv5aabbuIvf/kLF198Mb/97W+Ji4tj27ZtvP/++7z++utERERwzTXXcOeddxIfH09SUhL33nsvNlvnrPeuwl1EREREfHo/ahGfExYPgaGuD6g6S2Co63W9lJ2dTVVVFQMHDvT0i4OrcD906JBn2ziAN998k3vuuYcXX3yR0aNH88wzz3D22Wd7npOens7DDz/M7373O66++mquuOIK3nzzTR599FESExN58skn2bFjBzExMYwePZp77rnn2L/vH7nhhhtYtWoVF110EYZhcPHFF/PLX/6y0ZZxrTFhwgQ+++wzzjjjDOx2O7fccguLFy/mrrvu4vTTT6empoZevXoxbdo0T3H+9NNPe6bUR0ZGcscdd1BaWtru32NzDNM0zU55pWZ8/fXXPP3006xYsYJ9+/bxn//8h3PPPfeHcA39DD/2+9//3jPNIzMzk507dzZ6/Mknn2zUr9CSsrIyoqOjKS0tJSoqyvtvRERERDrdurxSznxpEU/8dBhZCc33Mnojp7CCe/7zPZ/eMoGh6dHtkNBP7F0Nr02EM19ov8L94Db49Fa4/itIG9k+5xSxSHV1NTk5OWRlZRESEvLDAyW7ofJg5wUJi4eYdpoVI53miL8/eFeHWjriXlFRwYgRI/j5z3/Oeeed1+Txffv2Nbr9xRdfcM0113D++ec3uv+RRx7xbBcAEBkZ2TGBRUREREREwFVEq5CWTmJp4T59+nSmT59+xMdTUlIa3f7kk0/Izs5utGw/uAr1Hx8rIiIiIiIi0hV0Tid9O9i/fz+fffYZ11xzTZPHnnrqKeLj4xk1ahRPP/009fX1Rz1XTU0NZWVljb5EREREREREfJHfLE731ltvERkZ2WRK/a9+9StGjx5NXFwcS5Ys4e6772bfvn0899xzRzzXk08+ycMPP9zRkUVERERERESOmd8U7m+88QaXXnppk4b+22+/3XN9+PDhBAUFccMNN/Dkk08SHBzc7LnuvvvuRs8rKyvz7FUoIiIiIiLyYxau6S1+rL1+b/yicF+4cCGbN2/mgw8+aPHYcePGUV9fT25uLgMGDGj2mODg4CMW9SIiIiIiIm6BgYEAVFZWEhoaanEa8TeVlZXAD79HbeUXhftf//pXxowZw4gRI1o8dvXq1dhsNpKSkjohmYiIiIiIdGV2u52YmBgKCgoACAsLO+K21SJupmlSWVlJQUEBMTEx2O32YzqfpYV7eXk527Zt89zOyclh9erVxMXF0bNnT8A1jf2f//wnzz77bJPnL126lGXLlpGdnU1kZCRLly7ltttu47LLLiM2NrbTvg8REREREem63DtYuYt3kdaKiYlplx3QLC3cly9fTnZ2tue2u+/8yiuv5M033wTg/fffxzRNLr744ibPDw4O5v333+ehhx6ipqaGrKwsbrvttkb96yIiIiIiIsfCMAxSU1NJSkqirq7O6jjiJwIDA495pN3N0sJ90qRJLTbrX3/99Vx//fXNPjZ69Gi++eabjogmIiIiIiLSiN1ub7dCTMQbfrOPu4iIiIiIiEh35BeL04mIiIhIg4PboeZQ+52vcEv7nUtERDqECncRERERf3FwO7w0umPOHahtrkREfJUKdxERERF/4R5pP/kOiM5ov/MGhkJUevudT0RE2pUKdxERERF/E50B8X2tTiEiIp1Ei9OJiIiIiIiI+DAV7iIiIiIiIiI+TIW7iIiIiIiIiA9T4S4iIiIiIiLiw1S4i4iIiIiIiPgwFe4iIiIiIiIiPkyFu4iIiIiIiIgPU+EuIiIiIiIi4sNUuIuIiIiIiIj4MBXuIiIiIiIiIj5MhbuIiIiIiIiID1PhLiIiIiIiIuLDVLiLiIiIiIiI+DAV7iIiIiIiIiI+TIW7iIiIiIiIiA9T4S4iIiIiIiLiw1S4i4iIiIiIiPgwFe4iIiIiIiIiPkyFu4iIiIiIiIgPU+EuIiIi0h2ZJlQVQ/FOqK+xOo2IiBxFgNUBRERERKQTmCbkLYd9a6E4x/VVXdrwoAERSRDdA6J7Qp9TIS7L0rgiIvIDFe4iIiIiXV3+97DyTTiwufH9hg0CQqCuEsr3u77yVsCG/0CP42HYhZA0yJLIIiLyAxXuIiIiIl1VUY6rYM9b4bodEAy9syG+n2tEPaYn2INdI++le6BsD+xdDTsXw57vXF/JQ2HM1ZA4wMrvRESkW1PhLiIiItIVbfocvn0FTCcYdug/FUZcDKGxTY8NjXF9pQyF/tOgLA/W/Qu2z4P96+CL38JxP4dBZ4NhdPZ3IiLS7alwFxEREelKTBNWvwNrP3Dd7jneNWIeldb6c0Slw4m/ghGXwPK/Qu5C+O4vULARTvoVBIZ1THYREWmWCncRERGRrsLpgKV/hG2zXbdHXOIaZW/rKHl4ApzyW0ga7Crgdy6C4lyYdDfE9mq32CIicnTaDk5ERESkK6ivhvmPuYp2wwbjb4aRlxz71HbDgEFnwdQnISze1Qf/xZ1QuLV9couISIs04i4iIiLSkQ5uh5pD7XOuwi3N3286YeGzrsXk7EGuUfKeJ7TPa7olDYKzXoT5j0PBBph9v6uY17ZxIiIdToW7iIiISEc5uB1eGt3+5w0MbXx73UewaynYAmDKI65F5jpCSDRMeQhm3Q+Fm13F+7QnITqjY15PREQAFe4iIiIiHcc90n7yHe1X3AaGuhaPc8tbASvfdl0f94uOK9o9rx8Gpz0MX94LRdtdl9P+D6JSO/Z1RUS6MRXuIiIiIh0tOgPi+7b/eQ/lw9dPAyb0m+rayq0zBEXAaY/AzLuhdBfMuhd+8mzzW82JiMgx0+J0IiIiIv6ovtrVb15bDgn9XaPtnSkkGqY+7tpmrqIAvvo/cNZ3bgYRkW5CI+4iIiLSpThNk50HK1m/t5T1e8uoqKknOjSQqNBAokICSYoKZlxWHGFBfv42aNmrUJzjKqAn3QP2wM7PEBoLpz4In90K+9fBirfg+Gs6P4eISBfn53+xREREpCPkFFZQUePbo6fbCsob3d5XWsW/VuxhzZ5SylvI/vbSnUwakMjUISkkR4V0ZMyOkb+2Ya92Aybe5dpv3SrR6XDSbbDgCdjwH0gcAJEp1uUREemCVLiLiIhIIzmFFWQ/s8DqGK1mM+DD5bv535q91DtNAEICbQxKiWJoejTxEUGUVdVTVl1HaVUd6/eWsrekmi/W5TNzXT5jesVyydieFn8XXnDUwTd/dl3vPw1ShlubB6DXiTD0fFj3L1j8Bzjp11YnEhHpUlS4i4iISCPukfabsvuSHhPawtHW2lZwiGdnbeFAeQ0Aw3tEc96oHvRJCifA1vxSPk7TZO2eUmau28eaPaUs31nM+r1lnDc6vdnjfc6G/0DpbgiJgdFXWp3mB6OugMKtrtkAK/5mdRoRkS5FhbuIiIg0Kz0mlKyEcKtjNMtpmvx96U6+XJ8PQHx4EFeMz+T4zFgMwzjqc22GwciMGEZmxJBXXMXri3awKf8Q7y7bBYCjYdTeJx3KhzUfuK4fdw0ER1ib53A2O5xyJ/zv11C+3+o0IiJdilaVFxEREb/icJq8smC7p2g/a3gqz/xsBGOz4los2n8sPTaUe38yiDOG/bAH+f2frKOwYQTfp5gmfPsqOGpc0+N7T7I6UVOhsTDxt0DDv8PuZZbGERHpKlS4i4iIiN+orXfywpwtLNxWiM2AX07qwyXjehESaG/zOQNsNi4/oRczjs8AYO2eUi58dSlFFbXtFbt97P4G9nwHtgA44Ubw8kOKTpM8FLJOdl3/+mmoLrU2j4hIF6DCXURERPxCdZ2D33+5ieU7iwm0G9x2Wn9O7pfYbucf3iMGgMSIYHYcqODnb35HZa2PrKzvqIVvX3NdH3IeRGdYm6clA890XVYcgFn3WZtFRKQLUOEuIiIiPs/hNHlm1mbW7y0jJNDGXdMGclyvuA55rUfOGUJ0aCCrd5dw8z9WUe9wdsjreGXLl64iODwRhl9odZqW2YMarhiw8u+wba6lcURE/J0KdxEREfF5/1i201O033vGYIakRXfYa2XEhfHGVccRHGBj3qYC7vnP95imhQvW1dfA9/90XR92IQT40b7zQ89zXf73V1BdZm0WERE/psJdREREfNqibYV8vs61EN2NE/vSN6njV1If0yuOP14yumGP+D08O2tLh7/mEW2ZCVVFrtH2vlOsy9EWx18HsZlQtgdmP2B1GhERv6XCXURERHxWTmEFr329HYBzR6YzNqtjpsc357TByTzx02EA/HH+NuZvKui01/aor4F1H7muD78I7IGdn+FYBIbCOS+7rq94E/ausjSOiIi/UuEuIiIiPqmsuo7nZm+mzmEyMiOGn43p0ekZZoztydUnZQJw50drOdjZ28RtmQlVxRCeBH0md+5rt5fMCa4p/pjw+W9d29qJiIhXVLiLiIiIz3GaJi/N20ZheS3JUcHclN0Xm82a7c/umjaQ/skRFJbXcPe/O7Hfvb76h9724Rf632j74U57GALDYc+3sPZDq9OIiPgdFe4iIiLic+ZtKmBdXinBATbuOG0AEcEBlmUJCbTzwkWjCLLbmLVhPx8u3905L7x5JlSXQESy//W2/1hUGpzyG9f12Q9AzSFr84iI+BkV7iIiIuJTiipq+ceyXQBcdHwGGXFhFieCwWlR3HF6fwAe/t8GcgsrOvYF66sb97bbrPvgot2MvwniekN5Pnz9tNVpRET8igp3ERER8RmmafLG4hyq6hz0TYpg6uAUqyN5XHtyb07oHUdlrYPbPlzdsfu7b5v3w2h7n1M77nU6U0AwTHvKdX3pn6Bwm7V5RET8iAp3ERER8Rnf7Chixc5i7DaD60/ubVlfe3PsNoNnLxxJZEgAq3aV8PY3OzvmhUwTNv3PdX3wOV1jtN2t/1Todzo462Dm76xOIyLiN1S4i4iIiE8or67nzaW5AJw7Ms0npsj/WHpMKHdPHwTA87O3dMwq8/vWQOluCAiFPn7e296cqU+CLRC2zYYdX1mdRkTEL6hwFxEREZ/wzrKdlFXVkR4Tyjkj062Oc0QXHZ/B4NQoyqrreXb2lvZ/Afdoe9/JEOR7H14cs4S+cNzVrutzH9b2cCIiraDCXURERCy3cV8ZX205gAFcf0pvAu2++xbFbjN46OwhALz37S7W7y1tv5Mfyofd37quDzyz/c7ra06507U9XN4K2PSp1WlERHye7/5VFBERkW7BNE3eXebqFz91YBL9kyMtTtSysVlxnDk8FdOEh/+7of32dt/0GWBC2iiI7tE+5/RFEUkw/peu63MfBafD2jwiIj5OhbuIiIhY6psdRWw/UEFwgI0LxvhPsXrPGYMICbTxbW4Rn67dd+wnrKuGbbNc1wedfezn83Un3gKhsVC4Gda8b3UaERGf1oWWKRURERF/U+9w8sFy157tZ41IIyYsyOJEsK2gvNXHnj+6B+8u28Uj/9tAanQIIYH2xgccqCXcmUJWa062Yz7UVkBkKqSP8SqzXwqJhgm3w+z7YcGTMPR8CAyxOpWIiE9S4S4iIiKWmbOxgP1lNUSHBvKTYamWZgkJdE1EvPWD1V4/90B5DRe8svQIjz7H/EMHyYo/ygkO3wJuwE/A6CaTIsdeB9/82bWK/vI3fpg+LyIijahwFxEREUtU1tbz71V7ALhgTI+mo9WdLDU6lOcuHEF1ndOr563ZU8IH3+0mJNDGnacPJDToh+8jb3cOLy8vp6K+hf3o87+Hkl0QEAL9uuAWcEcSGAqT7oL//RoWPgOjL4dg31/jQESks6lwFxEREUv8b81eDlXXkxYdQvaAJKvjAK7i3Vu94sNYvK2QPcVVbNhXygVjMn54sKSVb7W2fOG67J0NQRFeZ/BrIy+DJS/BwW3w3esw4TarE4mI+JxuMg9LREREfElRRS2ff58PwMVje2K3tTAi7cNshsH5o12L6n3+fT7lNfXenaC2HHZ947ref2o7p/MD9gDX9nAAS/4ItZXW5hER8UEq3EVERKTT/WdVHrUOJwOSIxnTK9bqOMdsbFYcGXFhVNU5+OJ7L1eYz1kIzjqI6QVxfTomoK8beoHr+68shJVvWZ1GRMTnqHAXERGRTlVcWctXWwoAuPD4DAzDf0fb3Vyj7ukAfLEun/JqL0bdt89xXfadDF3gZ9Em9gA4+XbX9cV/gPoaa/OIiPgYSwv3r7/+mrPOOou0tDQMw+Djjz9u9PhVV12FYRiNvqZNm9bomKKiIi699FKioqKIiYnhmmuuoby89du4iIiISOf6bO0+6hwmA5IjGZTSdRYiOz4zjp4No+6ftXbUvXQPHNjsWkW+d3bHBvR1Iy6GqHQ4tA9Wv2t1GhERn2Jp4V5RUcGIESN4+eWXj3jMtGnT2Ldvn+frvffea/T4pZdeyvr165k9ezaffvopX3/9Nddff31HRxcREZE2OFRdx5yN+wE4Z2Ralxhtd7MZBhc09LrPXL+PQ9V1LT9p+1zXZfoYCPX/loFjEhAMJ/3adX3R8+Boxc9PRKSbsHRV+enTpzN9+vSjHhMcHExKSkqzj23cuJGZM2fy3XffcdxxxwHw0ksvccYZZ/DMM8+QlpbW7plFRESk7Wauy6em3klmfBgjM2KsjtPujsuMpVd8GDsPVvLZ9/sYd7Ra3OmA7fNd1/tM7pR8Pm/0FfD1066t8b7/J4y8xOpEIiI+wee3g1uwYAFJSUnExsZy6qmn8thjjxEfHw/A0qVLiYmJ8RTtAFOmTMFms7Fs2TJ++tOfNnvOmpoaamp+6J0qKyvr2G9CREREqKyt58v1rpXkzx2Z3qVG292MhlH3Z2dvYdb6/Yw44Shbu+WvdS3GFhQBGWM7L6QvCwyFE2+B2Q/Awmdh+EVgs7seO7gdag617+sFR0J8N10QUET8ik8X7tOmTeO8884jKyuL7du3c8899zB9+nSWLl2K3W4nPz+fpKTG+74GBAQQFxdHfn7+Ec/75JNP8vDDD3d0fBERETnMnA37qah1kBYdwvGZcVbH6TCje8WSFh3C3tJqlu87yiJr7mnyWRPBHtQ54fzBcT93TZU/uA02fAxDz3cV7S+N7pjXu2WlincR8Xk+XbjPmDHDc33YsGEMHz6cPn36sGDBAiZPbvuUsrvvvpvbb7/dc7usrIyMjIxjyioiIiJHVlvv5LN1rg/Vzx6Zjs2P921vic0wOGN4Kq8vzGHJniMU7rUVsHOp63qfUzsvnD8IjoRxN8KCJ2DJSzDkvB9G2k++A6Lb6T1b6W7XqH57j+KLiHQAny7cf6x3794kJCSwbds2Jk+eTEpKCgUFBY2Oqa+vp6io6Ih98eDqmw8ODu7ouCIiItJg3qYCyqrqSIgI4qS+8VbH6XAn903kw+92U3KkbeFyF4GjxlWEJvTv3HD+4PhrYdFzsHcV7FwCQeGu+6MzIL6vtdlERCzgV/u479mzh4MHD5KamgrA+PHjKSkpYcWKFZ5j5s2bh9PpZNy4cVbFFBERkcM4nSafN2yPdtaINAJsfvX2o02CAmycNviHQQTT/NEB2+e5Lvt0473bjyY83rU9HMDSI+8+JCLSXVj6l7O8vJzVq1ezevVqAHJycli9ejW7du2ivLycO++8k2+++Ybc3Fzmzp3LOeecQ9++fZk6dSoAgwYNYtq0aVx33XV8++23LF68mJtvvpkZM2ZoRXkREREfsWJXMQfKa4gIDmBi/0Sr43Sa0wcnE9DwTmt9yWGTHCsKoWCD63rviZ0fzF+c8EvX5ebPXfvdi4h0Y5YW7suXL2fUqFGMGjUKgNtvv51Ro0bxwAMPYLfbWbt2LWeffTb9+/fnmmuuYcyYMSxcuLDRNPd3332XgQMHMnnyZM444wwmTJjAa6+9ZtW3JCIiIj8ys6G3/dSBSQQH2C1O03miQgMZneJadO7jXSE/PLBrCWBC4iAI7z4fZHgtsT/0nwaYrq3hRES6MUt73CdNmoTZZO7YD7788ssWzxEXF8c//vGP9owlIiIi7WTnwQo27CvDZrhGoLubkzKC+XZvLcsKA9lR4qB3jN3V3w6QOcHacP5g/E2wZSZs/sLqJCIilur6TWYiIiJiGfe+7cdnxhEf0f0Whk0Mc80wMDH46/e1jafJ9zrJwmR+IvNkSBnuWshPRKQbU+EuIiIiHaKsuo5F2woBmDb0yLu9dBf/2lJHyfZvXTeSBkN4grWB/IFhwPibf7jtOMIq/SIiXZwKdxEREekQ8zYVUOcwyUoIZ0BypNVxLNU7op7qevhoY6XrDo22t96Qn0JYw4cce1dam0VExCIq3EVERKTd1TudzN6wH4CpQ1IwuvmWZ2f0cE31frd0GE7TUOHujYAgGHq+63rOV83srSci0vWpcBcREZF2911OMUUVtUSFBnJin3ir41huYnINkfZ6csxUlkSfoWny3hp4puuyLA8ObLI2i4iIBVS4i4iISLubuX4fAFMGJRFo19uN0AA4L9Q1zfsd5+kWp/FDIVE/XN/8uXU5REQsor+kIiIi0q52Hqxgy/5y7IbBlEHdbwu45gTWFHNprWsv8tkHE8ivcFqcyI/lLoTqUqtTiIh0KhXuIiIi0q7mbSoA4LjMWGLDgixO4xuiDq6mvy2PscE7cZgG722stTqSf4rOAGc9bJ1tdRIRkU6lwl1ERETaTU29w7MF3KkDkyxO4zuiC13T5C/PqgDgvY111Dm0yJrXMie4Lrd8AU6HtVlERDqRCncRERFpN9/sKKKy1kFSZDBD06OtjuMTkigm7FAOYDB1dD8SQg0KKk3m7NSe5F5LGwVBEVC+X1vDiUi3osJdRERE2s38hmny2QOSsHXzLeDcptgbCszE/gRFxnPRwEAA3l6v6fJeswdB39Nc1zd9Zm0WEZFOpMJdRERE2sXuoko27z+EzYCJAxKtjuMzpthWuK5kjAPg4kFBGMCSvQ62l2i6t9cGTHdd5q2AQ/nWZhER6SQq3EVERKRdzNvsGm0f3VOL0rkFOKo4ybbedaOHq3DvEWnj1J4BAHy4qc6qaP4rKg3SRgOmq9ddRKQbUOEuIiIix6y23snCrQcAmDxIi9K5pZSsItioozYkAWJ6eu6/sGG6/L+2aJG6NhnwE9fl1tng0IcfItL1qXAXERGRY/ZtbhEVNQ4SIoIYnh5jdRyfkV78DQBlccPgsJ7/U3sGkBBqUFhlsmC3FqnzWo/jICweaspg9zdWpxER6XAq3EVEROSYzd24H4BJA5Kw2bQoHQBOB+nF3wJwKG5Yo4cC7QY/7ecadf9ws0aMvWazQ58pruva011EugEV7iIiInJM9pZUsSn/EIYBk/prUTq3yMJVhNSXUWKGUxHZp8nj7uny83bWU1Dp7Ox4/q9fw+rye1dBeYG1WUREOliA1QFERETEv321xdXbPrJHDPERwRan8R2xe+YAMN85EqMiCA40XUF+QJyNzUVOXllVy3n9A1t13vAgyIq2t2tWvxSZAinDIX8tbJ8LIy62OpGISIdR4S4iIiJt5nSaLNpWCMBEjbY3Ert7LgBzHGP47Nto+LbiiMe+sa6WN9a1fl/3+TPCVbyDa0/3/LWwbQ4MvwgMTSYVka5JhbuIiIi02bq9pRRV1BIebGd0r1ir4/iMkLIdhJVtx2kEMNW2jJNGDIaIpqvtV9ebPLmshjon/GJEED2jjl545pU7eXlVLRWtr/G7tl4nwrJXoHy/q4BPHWl1IhGRDqHCXURERNrs662u0fbxvRMItHeB0c6yPKirOubTxO74p+t00f1JPFhGYqQDopv/+ZyQZmfhHgebix1M7Km3Zl4JCIbeE2Hz565F6lS4i0gXpb8OIiIi0iaVtfV8l1MEwMT+CRanaQdlefDv69vlVHG2XDCguOggEOcqMI9gUkYAC/c4WJrn4IohJiEBWpXfK31PcxXuO5dATTkER1idSESk3alwFxERkTZZllNErcNJWnQIfRK7QLHkHmkffiGEN53W3loBdeVErvodAEWjboLwNAg/8gcbg+JtpIQb5FeYLNvnYGKG3p55Jb4vxGZCcS7kLICBZ1ocSESk/ekvg4iIiLTJ1w2ryZ/cPxHD6EKjxOFJEJ3e5qfH7FuAgUlFRCa1ScNbPN4wDE7pEcCHm+v4end91yrcC7d0/HkMA/qdDt++5pour8JdRLqgLvSXQURERDpLQVm1a+924OS+XWCafDuKKVwFQEnCmFY/Z0IPOx9urmPjQScHq5zEh/r5egGBoa7Lf1/XMef9saxJsPwNKNoORTsgrnf7vq6IiMVUuIuIiIjX3IvSDUmP1t7thzOdxBx0Fe7FCaNb/bTEMBsD42xsKnKyaI+Dc/r5eeEelQ4/fbVdFvrzCAx1nbc5IVGQMQ52Lobt81W4i0iXo8JdREREvGKaJgu3uqbJn9JPo+2HiyjbRmBdGfUBYZRHD/DquSf3CGBTUS0L99Rzdt8A/28/OFKR3VF6n+oq3HO+gjFXgU373ItI1+HnH+eKiIhIZ9ucf4iCQzWEBNo4PjPO6jg+xT1NvjRuOKbNu/GRcWl2Am2QV26SW2Z2RLyuLX00BEdCVRHkf291GhGRdqXCXURERLyyaJtrmvy4rHhCAjWqebiYgysBKIlvfX+7W3igwehk189z0Z76ds3VLdgDIfNk1/Ud863NIiLSzlS4i4iISKvVO5x8k3MQgAlalK6RgNoyIkq3AlCSMKpN5zi5h6twX5xXj8OpUXev9c52Xe5cAvXV1mYREWlHKtxFRESk1dbsKaWixkFMWCCDU6OsjuNToovWYOCkMqIntSFt+1BjRJKdiEAorYHvC53tnLAbSBwIEclQXwW7llmdRkSk3ahwFxERkVZbvN01Tf7E3vHYbH6+eFo7iyl0TZMvjm/9avI/FmAzODHd1Ruv6fJtYBg/jLpruryIdCEq3EVERKRVquscrMgtBuBETZNv7LBt4Eq82AauOe7p8t/tc1BVr+nyXus9yXW5dyVUlViZRESk3ahwFxERkVZZvrOYWoeTlKgQeieEWx3Hp4QdyiWotgSHPYRDMYOO6Vx9YmykhBvUOl3Fu3gpugfE9wPTCbkLrU4jItIuVLiLiIhIqyxuWE3+pL7x/r/HeDuLPbgCcG8DF3hM5zIMg5N7uKbLL9R0+bbpo+nyItK1qHAXERGRFpVV1bF2TwkAJ/bRNPkfc/e3lxxDf/vhTkp3TZdfX+ikpFrT5b2WeTIYNijcAqV5VqcRETlmKtxFRESkRd/kHMRpQlZCOGkxoVbH8Sn2unIiSzcDbd8G7seSw230ibFhAt/s06i710JjIa3h3yJngaVRRETagwp3ERERadGSbdq7/Uiii9ZimE4qw3tQE5rcbuc9sWHUfWme+tzbJGui6zJ3IZiatSAi/k2Fu4iIiBzVgUPVbN5/CAM4oXe81XF8jmc1+fj2GW13OyHNjgFsKXZyoFJ7unst4wSwBULpHijZaXUaEZFjosJdREREjmrJdtdo++C0KOLCgyxO42NMk+iDqwEojR/ZrqeOC7ExKN71Vm3pXo26ey0oDNLHuK5rdXkR8XMq3EVEROSo3IW7FqVrKqQqn5DqAziNAMpih7T7+U9Md60uvyRPfe5tkjnBdZm7SNPlRcSvqXAXERGRI8orqWJXUSV2w2BsZpzVcXxO9ME1AByKGYDTHtLu5x+basduwM4yk7xDmi7vtYyxYA+CsjwozrE6jYhIm6lwFxERkSNatsM12j6sRzQRIQEWp/E90UWrASiNG9Eh548MMhie6Hq7tmSvRt29FhgG6ce5rmu6vIj4MRXuIiIickTfNBTuJ/TWaHsTpoPoou+B9u9vP9wP0+UdmJru7T1NlxeRLkCFu4iIiDRrf1k1u4ursNsMjuulwv3HIsq2E1BfQX1AGOWRfTrsdY5LsRNkg/wKk73lKjy91uN4sAfDoX1QtN3qNCIibaLCXURERJr1fV4pACN6RBMerGnyPxZd5OpvL40bDjZ7h71OSIDB6BTX+dce0OryXgsMdRXvoOnyIuK3VLiLiIhIs9yFu/Zub557YbrSuOEd/lonpqlwPyaaLi8ifk6Fu4iIiDTrwKEaAu0GY3rFWh3F59gc1USWbAKgNG5kh7/eiCQ7oQFQWtvhL9U19TgOAoKhfD8c3Gp1GhERr6lwFxERkSMa0SOGsCBNk/+xqOIN2Mx6akISqQ5L7fDXC7IbHJfScdPxu7yAEOgx1nU9R9PlRcT/qHAXERGRRg5fuVzT5Jvn7m8viRsBhtEpr3lC2g8foDg13dt7mSe7Lnct0XR5EfE7KtxFRESkkZzCCgACbAaje2qafHM8/e3xHbN/e3OGJdgIbhh031Tk7LTX7TLSR7tWly/fD0U7rE4jIuIVFe4iIiLSyKJthQAMSIkkNEjTs38ssKaE8PJcoHMWpvO8rt1gcLzrrdvCPfWd9rpdRkCIq3gH2LXU2iwiIl5S4S4iIiIepmmycKurcB+WHm1xGt/kniZfEZlFfVDn/oyGJbg+SFmc59B0+bboOd51qcJdRPyMCncRERHx2JR/iH2l1YBrxF2aii5aCzT0t3eyvrGut25F1SYr8rU1nNd6jAXDDiU7ofyA1WlERFpNhbuIiIh4fLEu33M9OEDT5JswTU/hXmpB4R5g+2EhvM92aLq814IjIGWY63r+WmuziIh4QYW7iIiIeHx5WOEuTQVX5RNcfQCnEcCh2EGWZvl8R52my7dFzxNdlyrcRcSPqHAXERERAHYcKGfz/kPYbZ2zvZk/ii76HoDy6P447SGW5QgPhIJKk+WaLu+9nuNclyU7rc0hIuIFFe4iIiICwJfr9wMwvIcWpTsSd+FeGjfM0hwnpLr2dP9se52lOfxSWDwkDrA6hYiIV1S4i4iICAAz17umyZ/UJ8HiJD7KNIkqdhXuZbHWFu4TerjWH/g8px6HU9PlveZeXV5ExE+ocBcRERH2llSxZncJhgHjesdZHccnhVbsIai2BKctiEMx1o7YjkyyExUEBzRdvm0OL9xrDlmXQ0SklVS4i4iICF82jLYf3yuO2LAgi9P4Jvdq8mUxgzBtgZZmCbQZnJbpyjAzR6vLey0qHSJTXdd3LrE2i4hIK6hwFxEREWY2rCY/dWiKxUl8l3uavNX97W7Tslx97l/m1mFqdXnvubeFy11obQ4RkVZQ4S4iItLNFZbX8F1uEQBThyRbnMZHmU6ii9YB1ve3u53cI4CwANhbbvJ9odPqOP4nZbjrcve3UFtpbRYRkRaocBcREenm5mzYj9OEYenR9IgNszqOTwo7lEtAfTn19lDKo/paHQeAkACD7J6uUfeZO7S6vNei0l2XjhrI+draLCIiLVDhLiIi0s25V5OfpmnyR+Tubz8UOwRsdovT/GBq1g997pou7yXD+OH6li+syyEi0goq3EVERLqxsuo6Fm8rBGDqEBXuRxLtY/3tbtk9AwiywY5SJ9tKNF2+zbZ8CfrgQ0R8mAp3ERGRbmzexgLqHCb9kiLomxRhdRyfZDjriSpeD0Cpj/S3u0UGGUzo0TBdXqvLt01AKBzaB/tWW51EROSILC3cv/76a8466yzS0tIwDIOPP/7Y81hdXR133XUXw4YNIzw8nLS0NK644gr27t3b6ByZmZkYhtHo66mnnurk70RERMQ/uVeT1zT5Iwsv24bdUU19QASVkZlWx2nCvbr8zBz1ubdJj+Ncl5tnWptDROQoLC3cKyoqGDFiBC+//HKTxyorK1m5ciX3338/K1eu5N///jebN2/m7LPPbnLsI488wr59+zxft9xyS2fEFxER8WtVtQ4WbCkANE3+aH6YJj8UDN+brDilVwA2A9YXOtldpunyXut1outSfe4i4sMCrHzx6dOnM3369GYfi46OZvbs2Y3u++Mf/8jYsWPZtWsXPXv29NwfGRlJSorecIiIiHjjqy0HqK5z0iM2lCFpUVbH8VlRRe7CfbjFSZoXF2pjXKqdpXsdfJlbx7XDg62O5F96jgcM2LcGyvZCVJrViUREmvC9j42PorS0FMMwiImJaXT/U089RXx8PKNGjeLpp5+mvv7oPV41NTWUlZU1+hIREeluvnSvJj8kBePwFbbFw3DWEVWyCfCd/dubM+2w1eXFS6GxP0yX3/KltVlERI7Abwr36upq7rrrLi6++GKion4YFfjVr37F+++/z/z587nhhht44okn+O1vf3vUcz355JNER0d7vjIyMjo6voiIiE+prXcyZ+N+QP3tRxNRugWbs5baoGiqwntYHeeITs90TaJcke+goELT5b3Wf6rrcov63EXEN/lF4V5XV8eFF16IaZr8+c9/bvTY7bffzqRJkxg+fDi/+MUvePbZZ3nppZeoqak54vnuvvtuSktLPV+7d+/u6G9BRETEpyzdcZBD1fUkRgYzumes1XF8VnTROgDKYoc23vfbx6RG2BiZZMcEZu3UqLvX+je0bu5YALWVlkYREWmOzxfu7qJ9586dzJ49u9Foe3PGjRtHfX09ubm5RzwmODiYqKioRl8iIiLdycx1+wA4fXAyNpvvFqRWiyo5rHD3ce7V5b/U6vLeSx4C0RlQXw05X1mdRkSkCZ8u3N1F+9atW5kzZw7x8fEtPmf16tXYbDaSkpI6IaGIiIj/cThNZq13TZOfPjTV4jS+y3DWEVmyGfCPwn1qQ+G+dK+DkmrT4jR+xjCg/zTX9c1aXV5EfI+lq8qXl5ezbds2z+2cnBxWr15NXFwcqampXHDBBaxcuZJPP/0Uh8NBfr5rEZ24uDiCgoJYunQpy5YtIzs7m8jISJYuXcptt93GZZddRmyspv2JiIg0Z3luEQcraokODWRc7zir4/gsf+lvd8uKtjMwzsamIidzd9Vxfv8gqyP5lwHT4Lu/uBaoczrB5tPjWyLSzVhauC9fvpzs7GzP7dtvvx2AK6+8koceeoj//ve/AIwcObLR8+bPn8+kSZMIDg7m/fff56GHHqKmpoasrCxuu+02z3lERESkqZkNq8lPGZRMoF3FyZFEFa8HoCx2iE/3tx9ualYAm4pqmZlTr8K9tQq3uC6DoyAwFMrz4fsPIXFg284XHAnxfdovn4gIFhfukyZNwjSPPJXraI8BjB49mm+++aa9Y4mIiHRZpmny5bqGbeC0mvxRRRX7T3+729SsQP6wopavd9dTUWcSHugfHzhYIjDUdfnv65o+9p8bju3ct6xU8S4i7crSwl1EREQ61/d5pewtrSYsyM7J/RKsjuOzXP3tvr9/+48NirPRM8pgV5nJV7vrOaN3oNWRfFdUOvz0Vair+uG+nUvh+w8gphdMuM37c5buhoXPQs2h9sspIoIKdxERkW5lZsNo+6QBiYQE2i1O47siSrdid9ZSF+gf/e1uhmEwLTOQ19bWMjOnToV7S6LSG98OiXEV7iW7IDwJQrTzkIj4BjW2iYiIdCOzN7hWk586RNPkj8Yf+9vd3KvLz9tZT41Dq8t7JTzBNdqOCXtXWZ1GRMRDhbuIiEg3kVtYwdaCcgJsBpP6a9vUo3H3t5fG+U9/u9uoZDtJYQbldbAkr97qOP6nx3Guy7zl1uYQETmMCncREZFuYs5G12j72Kw4osM0hfpIDGc9kSUbAf9amM7NZhhMzXSNus/MUeHutfQxrsu9K8F0WptFRKSBCncREZFuYlbDNPnTBidbnMS3hVfsauhvj6IqPMPqOG0yLcv1wczs3HrqnZou75Wkwa4V56tL4eA2q9OIiAAq3EVERLqF4opalucWAa792+XIog+59vX2x/52t7GpdmKCDYqqTb7Ld1gdx7/YAiB1lOt63gprs4iINFDhLiIi0g3M21SA04SBKZFkxIVZHcenRZVtBfxzmrxboN1gSi/XdPkvNV3ee+7p8upzFxEfocJdRESkG3D3t5+uafJHZWASWb4D8M+F6Q43rWF1+Vm5dZimpst7xV24H9jimjIvImIxFe4iIiJdXHWdg6+2HABgigr3owqnqqG/PdJv+9vdJvQIICQA9pabbDioRda8Ep4AsZloWzgR8RUq3EVERLq4pdsPUlnrIDkqmGHp0VbH8WlRRiXg7m/377dJIQEGJ/dwjbrP2anp8l5Ld28Lpz53EbGef/9FEhERkRbNbpgmP2VQMoafLrbWWaI9hbt/T5N3O62hz31Obp3FSfzQ4YW7toUTEYupcBcREenCnE6TOdoGrlUMZz2RdK3CPbtnAAbwfaGTfeUqPr2SNBACw6CmDAq3Wp1GRLo5Fe4iIiJd2Pd5pRQcqiE8yM74PvFWx/Fp4aVbsRsmdfZwKiN6Wh2nXSSG2RiVbAdg7i5Nl/eKLQDSRrqua7q8iFhMhbuIiEgXNrthtH3igESCA+wWp/FtUUVrACiL6uv3/e2Hm+KZLq/C3WtpDavLa4E6EbFY1/mrJCIiIk3M1jT5Vos+uBaAssh+FidpX6dlugr3JXn1VNRpWzivpI1yXRZuhtpya7OISLemwl1ERKSL2nWwks37D2G3GWQPSLI6jk8znHVEFq8HoCyqv8Vp2lffGBu9ogxqnbBwt0bdvRKRBNE9XIvT7VtrdRoR6cZUuIuIiHRR7tXkj8+MJSYsyOI0vi384HrsjirqTDuVoalWx2lXhmEwpVcgALO1LZz33KPue1dam0NEujUV7iIiIl2UezX5KYM0Tb4lUfu/AeAQoV2qv91tSsN0+Xm76nE4NV3eK2mjXZd7V4Gpn52IWKPr/WUSERERSipr+Ta3CIDTB6dYnMb3Re1fBkCpGW5xko5xXLKd6GAorjZZsd9hdRz/kjzMtcJ8+X44tNfqNCLSTalwFxER6YLmby7A4TQZkBxJz/gwq+P4Nmc9UQXLASgzu+bPKtBukJ3hmi6v1eW9FBgCSYNd1/M0XV5ErKHCXUREpAuas6EAgCmDtShdSyKK1mOvr6A+MJJKgq2O02Hcq8vPUZ+79w6fLi8iYgEV7iIiIl1MTb2DBZtdhftpmibfInd/e1ncUMCwNkwHOiUjgEAb7Ch1sr1E0+W94l6gLv97cNRZm0VEuqUAqwOIiIhI+/pmRxEVtQ6SIoMZnh5tdRyf5+lvjxsBe7+yOE3HiQwyOCHNzsI9Dubk1tNnpN3zWE6pg4ra9n298CDIira3fKA/iMuCkBioLoEDmyBlmNWJRKSbUeEuIiLSxczekA/A5EHJ2GxddwS5XTjriXT3t8ePALpu4Q5wWq9AV+G+s54bRrraAnJKHWS/X9Ehrzd/RnjXKN4NG6SNhB0LXNPlVbiLSCdT4S4iItKFmKbp6W8/Tf3tLQovWk9AXTn1QVFURmZaHafDTe4VwAOLYcV+BwernMSH2jwj7TeNCiI9on26KPPKnby8qrbdR/EtlTaqoXBfCaOvsDqNiHQzKtxFRES6kHV5ZeSXVRMWZOfEPglWx/F5Ufu/BaAsaSwY/jMyvK3E2ebn9o62saPUyTvra5ncK9BzrvQIG1kxWv7oiFIb+twPbofqUghRG4qIdB4V7iIiIl2Ie5r8Kf0SCQn0n0LUKtHuhemSx1qcpHVCGt653Tqv6pjP9fyKWp5f8cOQeIjeFR5dWBzEZkJxLuxdDb0nWhxIRLoT/S9aRESkC5m90b0NXLLFSfyAs57Igu8AKE0ZD6bFeVohNcLGc9khVB/Djm55h5y8vLqWIBvcOz6YQJtBSIDr3NKCtNENhftKFe4i0qlUuIuIiHQRe4or2bivDJsBpw5Uf3tLGvW3xwyE4hyrI7XKsRbYmdEG/9hkUFxtUlkHI5NVsLda2ihY/2/YtxpMEwwt/iginUP/pxYREeki5m9yjbaP6RVLXHiQxWl8X3TDNnBlSWPB1n3aCgzDYEyy6/tdsV/7uXslaTDYAqHyIJTlWZ1GRLoRFe4iIiJdxJyGafKnDtQ0+daI8vS3j7M4Sec7vHA3TT/oEfAVAcGQNMh1fd8aa7OISLeiqfIiIiJdwIa9pSzeVghAz7hQ1uWVtvlc2wrK2yuW7zps//bSlBMsDtP5BifYCLZDcbXJjlInfWK6z4yDY5Y6AvLXugr3gT+xOo2IdBMq3EVERPxcTmEFZ7y4yHP7pn+sapfzhgR23Yl5P/S3R1MZO8jqOJ0uyG4wItHOt/kOVuY7VLh7I3UErHob8teA09Gt2ixExDoq3EVERPxcRc0PS4yP7x3PWSPSjvmcIYE2UqNDj/k8vqpRf7vRdT+gOJrRKa7CfcV+Bz8baHUaPxLfDwLDoLYCinZAQj+rE4lIN6DCXURExM85D+tRnjQgkayEcAvT+Ad3f3tpSvfrb3cblWTHAHaWmRyodJIY1j0/wPCazQ4pw2D3Mtd0eRXuItIJ9H9oERERP+fuSQ8KsDEoNcriNH7gsP72suTu19/uFhVs0D/O9VZwVYFWl/dK6gjXpRaoE5FOosJdRETEz32XUwRAv6QIAu36096SiEb97d17jvioJFd/9iptC+edlIbCvWADOGqtzSIi3YL+uouIiPi5b3NdhfvAFI22t4ZnG7hu3N/uNrJhW7j1hU5qHdoWrtViekJoLDhqoGCT1WlEpBvo3n+tRERE/Fx+aTXbD1QAMCAl0uI0/iEqX/3tbj0jDeJCDOqcruJdWskwIGW467qmy4tIJ1DhLiIi4sfmbSrwXI8I1pqzLXLWE6X+dg/DMBjVMOq+Wn3u3nH3ueevtjSGiHQPKtxFRET82NyN+62O4FciitZjr69Qf/thRiY1LFC334Fparp8q6WOdF0WboXaSkujiEjXp8JdRETET1XVOli0rdDqGH7F09+ePK7b97e7DU2wE2CDA1Ume8tVuLdaRBJEpoLphP3fW51GRLo4/cUSERHxU0u2F1JT7yQxItjqKH7D09+erP52t5AAg8Hx2hauTbQtnIh0EhXuIiIifmpuQ3/72Kw4i5P4B8NZp/72I9C2cG3kni6/b7WVKUSkG1DhLiIi4odM02TeRlfhfnxmrMVp/EP4wcP72wdYHcenuLeF21zkpLJO0+VbLWWY67JkF1QVW5tFRLo0Fe4iIiJ+aP3eMvLLqgkNtDO8R4zVcfyC+tuPLCXcRmq4gcOE7w9o1L3VQqIhro/ruqbLi0gH0l8tERERP+TeBm5CvwSCAvTnvDWi9i8D1N9+JO5t4dTn7iX1uYtIJ9BfehERET/k3gZuyqAki5P4B/W3t2xk0g/7uTu1LVzreQr31aCfm4h0EBXuIiIifqbgUDVr9pQCkD1AhXtrqL+9ZYPibYTYobQGckqdVsfxH0lDwBYAFQegUtszikjHUOEuIiLiZxZsOgDAiB7RJEWFWJzGP6i/vWUBNoNhie5RdxXurRYYAgkNHwYVbrE2i4h0WfrLJSIi4mfmNEyTP3VgssVJ/If621tnpLaFaxv3dPnCrdbmEJEuS4W7iIiIH6muc7Bom2s67mT1t7eKq7/9OwDKUtTffjQjk11vDXeUOCmtUb92q6WNdF2qcBeRDqLCXURExI98s+MglbUOkqOCGZIWZXUcvxB+cB32+krqgmKojFF/+9HEhdjoFWVgAmu0unzrJfSHgFCoq7A6iYh0USrcRURE/Ih7G7hTByZjGIbFafzDD/3tY9Xf3grubeFWq3BvPVsAJA+xOoWIdGH66yUiIuInTNNk7kZX4a5t4FovuqG/XdvAtc6ohj73tQccOJyaLt9q7j53EZEOoMJdRETET2zef4i8kiqCA2yc2CfB6jh+wXDWEenevz1FC9O1Rt9YGxGBUFEHW4q1unyrHV64O+qsyyEiXVKbCvfevXtz8ODBJveXlJTQu3fvYw4lIiIiTblH2yf0TSA0yG5xGv+g/nbv2QyDEUmaLu+12EwIinBdL9hgaRQR6XraVLjn5ubicDT9H3lNTQ15eXnHHEpERESacve3Zw/UNPnWUn9724zStnDeM2yQ0M91PW+ltVlEpMsJ8Obg//73v57rX375JdHR0Z7bDoeDuXPnkpmZ2W7hRERExKW4opZVu4oBFe7eUH9724xIsmMAuw+ZFFY6SQjThx6tEt8f9q6CvBVWJxGRLsarwv3cc88FwDAMrrzyykaPBQYGkpmZybPPPttu4URERMTlqy0HcJowMCWS9JhQq+P4BfW3t11EkEH/WBubi52sKnBwWqYK91Zxj7gXbICacgiOsDaPiHQZXv1f2Ol04nQ66dmzJwUFBZ7bTqeTmpoaNm/ezJlnntlRWUVERLotTZP3XkThGld/e3Cs+tvbYKS2hfNeeMOikaYDdi6xNouIdClt+vg0JyeHhAStZisiItIZ6h1OvtpyAIBTVbi3WnS+q3AqTRmv/vY2cPe5rzvgpNahbeG8lrvQ6gQi0oV4NVX+cHPnzmXu3LmekffDvfHGG8ccTERERFxW7S6htKqO6NBARmXEWB3Hb0TlLwWgLGW8xUn8U88og7gQg6Jqkw0HnYxM0k4GXlHhLiLtqE0fPz/88MOcfvrpzJ07l8LCQoqLixt9iYiISPtxT5Of2D+RALtGjlvDVl9F5IFVAJSmnGhxGv9kGAYjk1y/b5ou3wb71kB1qdUpRKSLaNOI+yuvvMKbb77J5Zdf3t55RERE5EfmNxTumibfepEFy7E5a6kJS6U6MtPqOH5rVLKdebscrN7vgKFWp/EjUelQlgc7l8KAaVanEZEuoE0f29fW1nLiifr0WkREpKPllVSxKf8QNsM14i6t80N/+4lgGBan8V9DEuzYDdhfabK/wtnyE8QlbaTrUtPlRaSdtKlwv/baa/nHP/5xzC/+9ddfc9ZZZ5GWloZhGHz88ceNHjdNkwceeIDU1FRCQ0OZMmUKW7dubXRMUVERl156KVFRUcTExHDNNddQXl5+zNlEREQ6Sk5hBevySlv19e43OwEYkBJJXklVs8dsK9DfvR/zFO6pGmg4FqEBBv3jXG8X1xzQdPlWSxvlulThLiLtpE1T5aurq3nttdeYM2cOw4cPJzAwsNHjzz33XKvOU1FRwYgRI/j5z3/Oeeed1+Tx3//+97z44ou89dZbZGVlcf/99zN16lQ2bNhASEgIAJdeein79u1j9uzZ1NXVcfXVV3P99de3ywcLIiIi7S2nsILsZxZ4/byN+w5x5kuLjnpMSKD63wHsNaWEF60HtDBdexiRaGfjQSdrCpycnml1Gj+RNtp1uW8tVBVDaKy1eUTE77WpcF+7di0jR44EYN26dY0eM7yYjjZ9+nSmT5/e7GOmafLCCy9w3333cc455wDw97//neTkZD7++GNmzJjBxo0bmTlzJt999x3HHXccAC+99BJnnHEGzzzzDGlpaW347kRERDpORU09ADdl9yU9JvSox9Y5nDz22QbqHCa3nNqX1OgjHx8SaDvq491J1P5lGKaTqqje1IalWB3H7w1PsvP+pjrWFzqod5oE2NR60KKweIjvBwe3uvZzH/gTqxOJiJ9rU+E+f/789s7RRE5ODvn5+UyZMsVzX3R0NOPGjWPp0qXMmDGDpUuXEhMT4ynaAaZMmYLNZmPZsmX89Kc/bfbcNTU11NTUeG6XlZV13DciIiLSjPSYULISwo96zOrdxdQ5TOLCgxjfO96rD8e7s0b7t8sx6xVlEBUEZbWwucjJkARtC9cqmRNchXvuIhXuInLMfHZOXX5+PgDJycmN7k9OTvY8lp+fT1JS4xV2AwICiIuL8xzTnCeffJLo6GjPV0ZGRjunFxEROXardpUAMCojRkW7F6Ib9m/XNnDtw2YYjGjYw32t+txbL+tk12WO+txF5Ni1acQ9Ozv7qG8g5s2b1+ZAneHuu+/m9ttv99wuKytT8S4iIj7FNE1P4T6yZ4ylWfxJYGUBYaVbMTEoSznB6jhdxvBEOwv3OFhT4ODiQVan8ROZDYX7/u+hsgjC4qzNIyJ+rU2Fu7u/3a2uro7Vq1ezbt06rrzyyvbIRUqKqydt//79pKameu7fv3+/5/VTUlIoKCho9Lz6+nqKioo8z29OcHAwwcHB7ZJTRESkI+SVVHGgvIZAu8HQtGir4/iN6P2u0faKuMHUB2tBsPYyPNGOAewsMymuNokN0QyQFkUkQcIAKNwMOxfDoLOsTiQifqxNhfvzzz/f7P0PPfRQu23FlpWVRUpKCnPnzvUU6mVlZSxbtowbb7wRgPHjx1NSUsKKFSsYM2YM4BrtdzqdjBs3rl1yiIiIWME92j4oNYqQQPUUt1b0Pld/e5mmyberqGCDzGgbOaVO1h5wMDGjTW8hu5+sk12Fe85CFe4ickzatcf9sssu44033mj18eXl5axevZrVq1cDrgXpVq9eza5duzAMg1tvvZXHHnuM//73v3z//fdcccUVpKWlce655wIwaNAgpk2bxnXXXce3337L4sWLufnmm5kxY4ZWlBcREb+2ancxAKMyNGrcaqZJlBam6zAjklxvG9Xn7gX3dHnt5y4ix6hdPy5dunSpZ3/11li+fDnZ2dme2+6+8yuvvJI333yT3/72t1RUVHD99ddTUlLChAkTmDlzZqPXePfdd7n55puZPHkyNpuN888/nxdffLH9vikREZFOVlFTz+b8QwCMUn97qwWX7yakIg+nEUBZ0vFWx+lyRiTa+XhrPWsLHDhNE5sWTGxZ5gTXZcEGqCiE8ARr84iI32pT4X7eeec1um2aJvv27WP58uXcf//9rT7PpEmTME3ziI8bhsEjjzzCI488csRj4uLi+Mc//tHq1xQREfF1a/eU4jQhLSaE5KjWfyDe3UXvWwxAeeJInIFH32pPvNc31kZoAJTXQU6pkz4xauFoUXgCJA12Fe65i2DIuVYnEhE/1aap8odvpRYdHU1cXByTJk3i888/58EHH2zvjCIiIt2Kpsm3Tcw+13Tk0tQJFifpmgJsBkMb9nBfU+C0OI0f0XR5EWkHbRpx/9vf/tbeOURERARwmiZrdpcAmibvFaeD6Ib+9hIV7h1mRJKd7/IdrC1wcF7/QKvj+IfMCfDtq64RdxGRNjqmHvcVK1awceNGAIYMGcKoUaPaJZSIiEh3teNAOWXV9YQG2hmQEml1HL8RcfB7AmrLqA+MpDx+uNVxuqzhia7JmltLnFTUmYQHqs+9RZkTAAMObILyAtc2cSIiXmpT4V5QUMCMGTNYsGABMTExAJSUlJCdnc37779PYmJie2YUERHpNtzbwA3vEU2ArV03f+nSove5RjNLU8aDTVuVdZTEMBtp4QZ7K0zWHXAwLk0/6xaFxUHyUNj/vWu6/NDzrU4kIn6oTe8IbrnlFg4dOsT69espKiqiqKiIdevWUVZWxq9+9av2zigiItJtrNI0+TaJcRfuaSdbnKTrG57U0OeubeFaL8vd567p8iLSNm0q3GfOnMmf/vQnBg0a5Llv8ODBvPzyy3zxxRftFk5ERKQ7Ka6sJaewAoARPWKsDeNHbHXlRBxYCai/vTOMSHQV7msLnEfdHUgO494WLkcL1IlI27SpcHc6nQQGNl2QJDAwEKdTq4yKiIi0xeqGafJ9EsOJCQuyNowfidq/DJtZT3VET2oie1kdp8sbFG8j0AYHq03yylW4t0qvEwEDDm6Fsn1WpxERP9Smwv3UU0/l17/+NXv37vXcl5eXx2233cbkyZPbLZyIiEh34t4GbqS2gfNKzF7X9OOS1JMsTtI9BAcYDIxzvYVcW6Dp8q0SGgupDYsm7lxsbRYR8UttKtz/+Mc/UlZWRmZmJn369KFPnz5kZWVRVlbGSy+91N4ZRUREurw6h5Pv80oB9bd7y7MwnabJd5oR6nP3nns/95yvrc0hIn6pTUuBZmRksHLlSubMmcOmTZsAGDRoEFOmTGnXcCIiIt3FpvxDVNc5iQ4NJCsh3Oo4fiOoYi9hZdsxDRulKSdaHafbGJ5khw11bDzopNZhEmTXtnAtyjwZlv7RtbK8iIiXvBpxnzdvHoMHD6asrAzDMDjttNO45ZZbuOWWWzj++OMZMmQICxfqf0YiIiLeWrXLPU0+BpuhIqi13KPt5fHDcQRHW5ym++gRYRAXYlDnhA0Htb5Rq/QaD4YNinZAaZ7VaUTEz3hVuL/wwgtcd911REVFNXksOjqaG264geeee67dwomIiHQX7v3bNU3eO55t4FK1DVxnMgyDEUnqc/dKSDSkjnBd17ZwIuIlrwr3NWvWMG3atCM+fvrpp7NixYpjDiUiItKd7CutIr+sGrvNYFi6Ro1bzXQSvW8JoIXprDA8UX3uXnP3ueeqz11EvONV4b5///5mt4FzCwgI4MCBA8ccSkREpDtxj7YPTIkkLKhNy890S+FFGwisKcIREE554iir43Q7QxPsGMDecpPCSk2Xb5WsU1yXGnEXES959e4gPT2ddevW0bdv32YfX7t2Lampqe0STEREpLtYtbsEgFHaBs4rntXk44ZhFu889hOW7j72c3QjEUEGfWNtbC12suaAk8m92rRZUffS8wQw7FCcCyW7ISbD6kQi4ie8KtzPOOMM7r//fqZNm0ZISEijx6qqqnjwwQc588wz2zWgiIhIV1ZV62DjvjJA/e3eitk1C4DS/Bz436/b78QBwe13ri5uRJKdrcVO1hY4mNxLs0VaFBwJaaMgb7lrdfmRl1idSET8hFf/h73vvvv497//Tf/+/bn55psZMGAAAJs2beLll1/G4XBw7733dkhQERGRrmhdXikOp0lyVDCp0SEtP0EAsNceIrLoewCKB1wIsQPb58QBwRCe0D7n6gZGJNr4aDN8X+ig3mkSYNOOCC3KOtlVuOeocBeR1vOqcE9OTmbJkiXceOON3H333ZimCbhWFp06dSovv/wyycnJHRJURESkK1q127UN3KiMWAxtA9dq0fmLsZkOqswgamIHQnS61ZG6pd4xNiICobwOthU7GRhvtzqS78ucAIueh53qcxeR1vN6TlOvXr34/PPPKS4uZtu2bZimSb9+/YiNVV+eiIiIN0zT/KG/XdPkvRKT9xUAJWa4xUm6N5thMDTRzjd7HXxf6FDh3hoZJ4AtAEp2QfFOiO1ldSIR8QNtXkUkNjaW448/nrFjx6poFxERaYPcg5WUVNYRHGBjUGqU1XH8h2kSs9dduEdYHEaGJbiK9e8PaGX5VgmOgLTRrutaXV5EWknLf4qIiFhk1S7XNPlh6dEE2vUnubXCSjYTXJmP0xZEGWFWx+n2hiW6fne3FTupqDMtTuMnMie4LlW4i0gr6V2CiIiIRdzT5EdqmrxX3KPtpfEjcOqtjOUSw2ykhhuYwIZCh9Vx/IMKdxHxkv7aiYiIWKC8pp7tBeWA9m/3Vkze1wCUJB5vcRJxG5bomi6/VtPlWydjnKvPvbShz11EpAUq3EVERCywZf8hTKBXfBhx4UFWx/EbtrpyIg8sB1S4+xL3dPl1GnFvHfW5i4iXVLiLiIhYYHP+IQBG99Rouzei9y3B5qyjKrIX1eHaAs5XDI63YzMgv8LkQKVG3Vsl62TXpQp3EWkFFe4iIiIW2FrgKtxHZcRYG8TPxLpXk0+bZG0QaSQs0KBvjOttpabLt5L63EXECyrcRURELFBd5yQyJIA+idrOrNVMk5i8BQCUpJ9ibRZpwj1d/vsDmi7fKo363HOtTiMiPk6Fu4iIiEVG9ojBZjOsjuE3Qku3Ely5z7UNXPIJVseRH3EvULe+0IHT1LZwLQoKh/QxrusadReRFqhwFxERscgobQPnlZg81zT5spQTcAaEWpxGfqxPjI3QACivg73lKtxbRdPlRaSVAqwOICIi0p3kl1YDYDNgeI8Ya8P4mdi9CwAoTptobRBpVoDNYEi8neX7HWwr7uZ97oVbWndcZKrrcvs8yFsFRjMzcIIjIb5P+2UTEb+kwl1ERKQTfZdbBECv+HDCg/VnuLXstWVE7v8OgJL0bIvTyJEMS7SxfL+DrSXdtHAPbJgJ8u/rvHte+X74y6QjP37LShXvIt2c3jGIiIh0ouW5xQAMSI60OIl/icn7CptZT2VUH6qjMq2OI0fg6nOvY1dZNy3co9Lhp69CXVXrn7P4D1CcAyMudi1Yd7jS3bDwWag51L45RcTvqHAXERHpJJW19azNKwFgQIoKd2/E7pkLQHHGZIuTyNGkhBskhBoUVnXjHveodO+O7zHWVbiX74f4vh2TSUT8nhanExER6SRLth2kzuEqaJIigy1O4z8MZ90P/e09VLj7MsMwPNvCSSulDHNd5n8PWo1fRI5A/2cVERHpJHM3FXiuG80tQiXNiixYQUBtGXXBsRxKGG11HGmBe1s4aaXEga793CsOuEbdRUSaocJdRESkE5imyYLNBS0fKE14psmnZ4NNRaGvG5pgx/2x1MGqbtrr7o3AEEjo77qev9baLCLis1S4i4iIdIKN+w6xr7Sa4AD96fWKaRK7Zw4AxT1OtTiMtEZkkEFahKt0X13gsDiNn0g+bLq8iEgz9O5BRESkE8xvGG0fkRFjbRA/E1K2g9BDO3HagihJO8XqONJKfWNcbzFVuLeSu899/zr1uYtIs1S4i4iIdIJ5Df3tx2fGWpzEv8Q1TJMvSx6HMzDC4jTSWn1j3YW7E1OFaMuS1OcuIkenwl1ERKSDFVXUsnKXa//243rFWZzGv/wwTV6ryfuTXlGut5jFNSabi9Tn3qIA9bmLyNGpcBcREelgX20pwDRhYEokidoGrtUCqouIPLASgCIV7n4lwPbDrgmL8uotTOJHUtTnLiJHpsJdRESkg83bdACAyYOSLE7iX2LyFmCYTipiB1IbkW51HGmjhXtUuLeK9nMXkaNQ4S4iItKB6h1OvmpYmO7UgSrcvRHn2QZOo+3+bNk+BzUOFaItcu/nXlkIh/KtTiMiPkaFu4iISAdauauEsup6YsMCGZmhhelay3DUEL1vIQDFGVMsTiNtFRtsUF0PK/K1unyLDu9z36/p8iLSmAp3ERGRDuReTX5i/0Tsh/X9ytFF71tMQF05taFJlMcPszqOtNHIJNdbTU2XbyX1uYvIEahwFxER6UDzNrm2dsrWNHmvxO/6EoCijNPB0NsVfzUyyQ7AIhXuraM+dxE5Av0lFBER6SB7iivZsr8cm+EacZdWctZ7toE72HOaxWHkWIxqKNzXFToprta2cC1Sn7uIHIEKdxERkQ4yv2Ga/HG94ogJC7I4jf+I2v8tgTXF1AXHUpY81uo4cgziQm0MiLNhAovz1OfeooAQSBjguq4+dxE5jAp3ERGRDuLub9c0ee/E75oJQHGPKa7RR/FrE9Jd/4aaLt9Knunya63NISI+RYW7iIhIB6iqdbBk+0FA28B5xXQSt3sWAAd7aZp8VzChh2u6/MI99Zjq226Z+txFpBkq3EVERDrA0h2F1NQ7SY8JpX9yhNVx/EbEgVUEVRVQHxhBacqJVseRdjAuNYBAG+SVm+wsUyHaosQBDX3uB1297iIiqHAXERHpEHM3uqfJJ2IY2gauteJ3u1aTL04/FdMebHEaaQ9hgQajkxtWl8/TdPkWHd7nfnCbtVlExGeocBcREWlnpml6FqbTNHkvmCZxO1397UWaJt+lTOihPnevuKfLq3AXkQYq3EVERNrZ5v2H2FtaTUigjRP7JFgdx2+EFW8gpGIPDnsIJamnWB1H2tGEdNeI+5K8ehxOTZdvkQp3EfkRFe4iIiLtzL2a/Il9EggJtFucxn/EN4y2l6RPxBkYZnEaaU/DEu1EBkFZLXxfqP3cW+Tez7261OokIuIjVLiLiIi0s/naBq5N4hr624syplqcRNpbgM1gfJpruvxiTZdvWUCwq3gXEWmgwl1ERKQdlVTWsmJnMaD+dm+Elm4jrHQbTlsgxT1OtTqOdICT3X3uWqCuddzT5UVEUOEuIiLSrr7acgCnCQOSI0mPCbU6jt+Iz/0UgNLUk3AERVmcRjrCSQ197ivyHVTVqc+9RcmHFe7az12k21PhLiIi0o7maZq890yThNz/AVCYebbFYaSjZEXbSI8wqHXCt/kOq+P4vsQBYGtYI6Msz9osImI5Fe4iIiLtxOE0+WrLAQAmD1Lh3lphxRsILcvBaQ+mKGOK1XGkgxiGwUnp6nNvtYBgiMl0Xd+7ytIoImI9Fe4iIiLtZNWuYkoq64gODWRURozVcfxGQo5rtL04/VScgREWp5GOdFJDn/tC9bm3Tnxf1+W+1ZbGEBHrqXAXERFpJ+5p8hP7JxJg15/YVjGdJOx09bcXZp5pcRjpaO4+940HnRRWaVu4FrkL972r1ecu0s3pXYWIiEg7cRfuWk2+9SIPrCS4Yi/1gRGUpGdbHUc6WEKojUHxrrefi/PU596i2F6uy8pCKNphbRYRsVSA1QFERER8WU5hBRU1LU/rLThUzab8Q9gMSIoMZl1eabPHbSsob++Ifs29mnxxxmk4A0IsTiOdYUJ6ABsP1rJ4Tz3n9A20Oo5vswf9cD3na4jvY10WEbGUCncREZEjyCmsIPuZBV49x2nCJa8va/G4kEBNesNZT/zOzwAozDzL4jDSWSb0COAva2tZlFePaZoYhmF1JP+QuwiOu9rqFCJiERXuIiIiR+Aeab8pu2+Le7L/fWkum/IPcfrgZCYNOPpU+ZBAG6nR2uM9ev83BFUfpC44ltLUk6yOI51kbIqdIBvsLTfJKXXSO8ZudST/kLvI1eeuDzpEuiUV7iIiIi1IjwklKyH8iI/X1jvZcaACcPW394o/8rHyA/c0+YM9p2HaNGW6uwgNNBiTYmfpXgeL8xwq3FvDHgTl+XBwOyT0tTqNiFjA5+fpZWZmYhhGk6+bbroJgEmTJjV57Be/+IXFqUVEpDtZl1dKrcNJQkQQPePCrI7jFwxHDfG7ZgJwUNPku50JDfu5L9J+7q2TNNh1mfu1tTlExDI+X7h/99137Nu3z/M1e/ZsAH72s595jrnuuusaHfP73//eqrgiItINrdhVDMDonrHq122lmL2LCKgtozY0mbKk462OI53MvZ/7kr311Du1zVmL0ka5LnMWWptDRCzj81PlExMTG91+6qmn6NOnDxMnTvTcFxYWRkpKSmdHExERwWmarGwo3Mf0irU4jf9IyPkPAIW9zgCbpkp3N8MSbEQFQVktfH/Awahkn39Laq20UbDib66V5dXnLtIt+fyI++Fqa2t55513+PnPf95oROPdd98lISGBoUOHcvfdd1NZWXnU89TU1FBWVtboS0REpC1yCisoqawjNNDOoNQoq+P4BXtNKXG75wBwoM95FqcRK9htBic2TJfXfu6tkDQYAkJd+7kXbLQ6jYhYwK8K948//piSkhKuuuoqz32XXHIJ77zzDvPnz+fuu+/m7bff5rLLLjvqeZ588kmio6M9XxkZGR2cXEREuqoVO12j7cN7RBNo96s/q5ZJ2PkpNmctFTEDqIwdbHUcschJDYX7QvW5t8weCL3Gu67nqM9dpDvyq3lJf/3rX5k+fTppaWme+66//nrP9WHDhpGamsrkyZPZvn07ffr0afY8d999N7fffrvndllZmYp3ERFpE3fhrmnyrZe4/V8AHOhzvqb8dmMnN/S5r9zvoLLOJCxQvwtHlTURts+DnK/gBC3ELNLd+M3QwM6dO5kzZw7XXnvtUY8bN24cANu2bTviMcHBwURFRTX6EhER8daBQzXsKqrEMGBkRozVcfxCSOkOIgtXYxp2CrPOsTqOWKhXlEF6hEGdE5bt06h7i7JOcV3mLgKHfl4i3Y3fFO5/+9vfSEpK4ic/+clRj1u9ejUAqampnZBKRES6M/eidAOSI4kM0T7krZG4wzXaXpJ2CnWhiS0cLV2ZYRhM6KE+91ZLHQHB0VBTBvlrrE4jIp3MLwp3p9PJ3/72N6688koCAn6Y3b99+3YeffRRVqxYQW5uLv/973+54oorOOWUUxg+fLiFiUVEpDvQNHkvOR0k7vgYgII+51ubRXyC9nP3gs0OmRNc19XnLtLt+EWP+5w5c9i1axc///nPG90fFBTEnDlzeOGFF6ioqCAjI4Pzzz+f++67z6KkIiLSXVTW1rNhn2tXkjE9Vbi3RvSO/xFcuY/6wEiKQzPh4JHb2lqldHe75BLrnJju2gpwU5GTA5VOEsP8YkzJOlmnwObPYMdXMOE2q9OISCfyi8L99NNPxzTNJvdnZGTw1VdfWZBIRES6u7V7SnE4TdKiQ0iNCbU6ju8ryyNx8UNgg8IaO+bnd7bfuQOC2+9c0qniQ20MSbCxvtDJkjwH5/RT4X5UvSe6Lnd9A/U1+t0X6Ub8onAXERHxNe5p8qM1Tb5V7JWFxBmHADgw9FqIyGqfEwcEQ3hC+5xLLDEhPYD1hbUsyqvnnH5aK+KoEgdCeCJUHIA9yyHzJKsTiUgnUeEuIiLiJYfTZNVu9bd7Iz7/a+yGSWVIMuVpJ2kbOPE4KT2AV9fUsmhPPaZpYuh348gMwzVdft2/XH3uKtxFug3NRxIREfHS5v2HqKhxEBEcQP+kSKvj+IXEPbMBOJBwgop2aWRsqp0gO+yrMNlR6rQ6ju9zbwuXo3ZRke5EhbuIiIiXVjZMkx/VMwabTUVoS0JLthBVvA7ThAMJY62OIz4mJMDguGTXInWL9mhbuBZlNfS57/kOaiuszSIinUaFu4iIiBdM09Q2cF5K3vo+AMVEUBcUY20Y8UknNeznvihP28K1KDYTonuCsx52LbU6jYh0EhXuIiIiXthbWk1+WTUBNoPh6TFWx/F5tvpqEnf8G4D9Tn3QIc07uaFw/2ZvPfXOpjsJyWHcfe6g/dxFuhEV7iIiIl5wj7YPTosiNMhucRrfF7/zMwJqy6gOTaaEcKvjiI8aEm8jOhgO1cKaAk2Xb5EKd5FuR4W7iIiIF1ZqmrxXkhqmyRdknAFoPQBpnt1mcFK6a9R9cZ4K9xa5C/e9q6Gq2NIoItI5VLiLiIi0UllVHVsKXHuRj+mpwr0locWbiTqwAqcRQEHGVKvjiI9zF+7qc2+FqFRI6A+YkLvY6jQi0glUuIuIiLTSqt0lmCZkxocRHxFsdRyfl7z1PQCKMyZTFxxncRrxdRMaCvdV+x1U1KnPvUWaLi/SrQRYHUBERMRfuKfJj+4O0+TL8qCuqs1PtzmqSdz+LwD2J0+E0t3tlUy6qF7RNjIiDXYfMvl2Xz3ZPQOtjuTbsk6B715X4S7STahwFxERaYXaeidr9pQA3WCafFke/Pv6YzpFvFFCgK2CajOQ0m/+jqe/PUAzFeTIJqQH8N6mOhbtcahwb0nmyYABBzbCof0QmWx1IhHpQCrcRUREWmHDvjJq6p3EhgWSldDFV0d3j7QPvxDCk9p0iuT1T0MF7M84A9JOd90ZEAzhCe0UUrqik3o0FO7qc29ZWBykDIP8tZC7EIZdYHUiEelAKtxFRERaYcVhq8kbRjdZHT08CaLTvX9a6TYiK3JxGgEc6P1TCI5p/2zSJZ2YZscANhc5Kah0khSm5ZiOKusUV+Ge85UKd5EuTv83FBERaYFpmqzc1dDf3tWnybeD1N3/A+BgyknUqWgXL8SF2hiS4Hp7uniPRt1blDXRdak+d5EuT4W7iIhIC/aVVlNUUUtwgI0hadFWx/FpgdVFxOe7tqfa1/Msi9OIP/phWzjt596iXuPBsENxLhTvtDqNiHQgFe4iIiIt2LCvDIBh6dEEBehP59Gk7PkCm1lPWcwgKqL6Wh1H/NDJPVyF++K8ekxT28IdVXAkpI9xXdeou0iXpncfIiIiLdiw11W4H5epvciPxnDUkrznS0Cj7dJ2x6XYCbJDfoXJ9hKn1XF8X29NlxfpDlS4i4iItCC/rBqb0Q22gTtGCflfE1hXRk1IIkWJ46yOI34qJMBgbIodgIXqc29Z1imuy5yvQTMURLosFe4iIiKtMCg1iogQbcZyRKZJ6i7XonT5GWeAzW5xIPFnExqmyy/aoz73FvUYC/ZgKM+Hwq1WpxGRDqJ3ICIiIq1wvKbJH1VU8feEl+/EYQtmf/ppVscRP3dyjwCeWlbD0r311DpMguzdZAvGIynccvTHk4fA3pX/3959x0dR538cf832Te+VQELvvQgoiCLYu2Iv59kOPeud5fQ8PU89+/3Qs+LZu9hAsVCkI9JrQkJCQgrpPdvn98dAFCmhbDKb3c/z8djHluzOvgOT3fnMt8G692DAeW1vzxoJ8T38k00I0SGkcBdCCCEOoqbZ1Xp7ZDfpJn8oqYWzAahIm4TXHKFzGtHZ9Ys3kGBXqGxR+aXMy7j0ED1kNdu161nXH97zlz6vXQ7HrWukeBeiEwnRT0EhhBCibSt3VAOQHmMnPsKqc5rAZWsuJbZiFQClXc/UOY0IBgZF4YQuJj7f7mbxLk/oFu5R6XDeK+BuOfTzagq0gt0cBlMeBeUQo2HrimDxM+Bs8GdSIUQ7C9FPQSGEEEGrKs9vB6TLt1UAMCAtyi/bC1ZpBZ+joFITPxxHeBe944gAknsMs8J3j9aKz+/zPZzR3Uu4BbKiQ3DuhKj0tp8TmwU/vwruZkCBeFmKUYhgI4W7EEKI4FGVBzOG+2VTDaqd9c5XABP9o2Vm64OxOKpILJkPQHHWBTqnEYFi7zyOt89vo6X4MOTV+ThzVhMACy4JD83ivS0GI6QMhqIVULoWEnrpnUgI4WdSuAshhAgee1vaT7gLojOOaVMLiqx4Vmpfk0k2mdn6YFJ3folB9VAf04+G2AF6xxEBIjXCwLOTbDiO8ZzXjDVOSptUJncz8uNOL02utl8TstKGaoV7yVoYdLHeaYQQfiaFuxBCiOATnXHMXUW/W9MMSEv7oZhc9SQXfw9AcdaFOqcRgSY14thXHR6VauSrXA9VLbI+eZtSh2nX5VvB7QCzTd88Qgi/knXchRBCiN9xeFQWFknR3paUojkYvQ6aIrOojffPEAUhfmtwotYtfnvN0Y+VDxlRaRCeCD4P7N6kdxohhJ9J4S6EEEL8zrJiD01uiLdKsXAwBk8LqYVzACjOvACUEF9nW7SL3rEGrEZodOudpBNQFEjb0+peulbfLEIIv5PCXQghhPid7wq01vbjEmRA7cEk7/oOk6eRlrA0qpLH6h1HBCmzUaF/vByuHrbUodp1yTo9Uwgh2oF8EgohhBC/4fGpfL+ncB+bJIX7gSg+N2k7vwSgOPN8UGSWb9F+9naXF4chdQigQO1OaK7WO40Qwo+kcBdCCCF+Y2WJlxqHSqxNYVCMjHM/kMSS+VhcNTit8VSmTtQ7jghyg5N+LdwdHpmk7pBs0RDfQ7tduk7XKEII/5LCXQghhPiNb/K1wbRTM00Y5VtyP4rXRZcdHwNQknkeqsGscyIR7FLDFWKs2u2NlbI0Y5tau8vLOHchgokckgghhBB7eH0q3+VrreyndZeC9EBSds3F6qzCaUtgd5epescRIUBRFHrFaq3ua3dL4d6m1gnq1oMqPRSECBZSuAshhBB7rCrzUtmiEm2FcWkyrvb3DJ4W0vM/A2BX92nS2i46TK9Y7ZB1TbkU7m1K6g9GK7RUa2PdhRBBQQp3IYQQYo9vd2jd5E/pZsZslOXNfi+18GvM7jpawtIoTz1J7zgihPSI0Q5ZdzWoFDfIMo2HZDRD8gDttswuL0TQkMJdCCGEAHyqyrd7usmf3t2kc5rAY3I3kLbzCwCKelwKBumRIDqO3fTribTFu2TSyDalDdWuZT13IYKGFO5CCCEEsGa3l/JmlUgLjO8ihfvvpRV8gcnTTFNEJlXJ4/WOI0LYIinc25a6Z5z77k3gdeubRQjhF1K4CyGEEMA3O7RiYHI3M1bpJr8Ps7OGlMLZABT1vAwUOXwQ+lmyy4PXJ5OuHVJsJthiwOOE8i16pxFC+IF88wohhAh5PlVtHd9+mnST3096/icYfU4aontTkzBK7zgihIWbod4F6ytkkrpDUhRIH67dLlmjbxYhhF9I4S6EECLkrS/3UtqkEm6GCdJNfh/2xiKSd30HQGHPK7SCQAidDE3S5lZYVCSFe5vSRmjXxav1zSGE8Asp3IUQQoS8b/d0kz+pqwmbSQrTVqpKZs4bGFQv1YmjqY8brHciEeKG7SncZYK6w5A2FFCgpgCaq3QOI4Q4VlK4CyGECGmqqvJNvtZN/vTusi75b8XUbSamai0+xcTO3tfqHUcIhidrhfu6ci91Thnnfki2aEjopd0ulu7yQnR2UrgLIYQIaesrfOxqULGbYGKGdJPfS0Elc+dnAJR2OwtHWKrOiYSApDADPWIMeFVYXiyt7m1K39NdXsa5C9HpSeEuhBAipH2dq7W2n5JpIsws3eT3SlGqsTvLcVliKM66SO84QrQ6Yc88FD9Jd/m2pe2doG4t+GReACE6M2laEEIck/zKJpqc/j14CreayEoI9+s2hTgQn6oyO08r3M/qId3k9zI7a+iiVAJQ2PNKvKYwnRMJ8auJGUbe3ASLijyoqooiEyYeXEJvsESAqxEqcyCpn96JhBBHSQp3IcRRy69sYtLTC9tl2wvuPlGKd9HuVpV62d2sEmmBCdJNvlVGzpuYFB+NYV2pSJukdxwh9jEm1YTFAMWNKvl1PrrHGPWOFLgMRkgbBgWLte7yUrgL0WnJUYoQ4qjtbWmfPqkn6TF2v2yzuLaFFxfk+r0VX4gD+XpPa/vUTDNWo7TaAUSWryKpaC4ABd0uBEVG1YnAEmZWGJliZFmJl0W7vFK4tyVtuFa4F6+GoZfrnUYIcZSkcBcigPm7G3p7dUFPj7FL67jodDw+tXUZuLN6Sjd5AMXrpMfy+1BQKfdF0xDZQ+9IQhzQhAyTVrgXebhmoEXvOIEtfc8498rt4KjTN4sQ4qhJ4S5EgGqvbujSBV0IzfISL1UOlTibwrg0abED6LJhBvb6HbiscRQ0J+gdR4iDmpBh4omVTpaXeHB6Vekxcyhh8RCbqa3nXrIOotN1DiSEOBpSuAsRoPzdDV26oAuxr72zyZ/W3YRZDvoJq95C+uZXAMgfcAveVZ/onEiIg+sXZyApTKG8WWVVqZfju8gh7SGlj9hTuK+Wwl2ITko+5YQIcNINXQj/c3pV5ubLbPJ7KT43PZbfg6J6qep6GtUpxwNSuIvApSgKJ2aY+DjbzYJCjxTubUkbAZs+g+I10PdMvdMIIY6CzDgjhBAi5Cwu8lDvguQwhVEp0k0+dcvrRFRvxmOJJn/0P/SOI8RhmdRVK9YXFElPsjYl9QOTDRy1UF+idxohxFGQwl0IIUTI2Tub/Bk9zBgNod1N3l67nYz1/wEgf+SDuO2JOicS4vCMTzdhMsCOWh8763x6xwlsRjOkDtFul2/VN4sQ4qhI4S6EECKktLhVfijYM5t8j9DuXmvwtNB70a0YfC5q0iZS2f08vSMJcdiirNqycAALpdW9bekjtevyLfrmEEIcFSnchRBChJQfd3po9kCXSIWhSaHdTT5z1SOE1eXgsieSN+5JUEK794HofCZl7OkuXyiFe5u6jNKuawp0jSGEODpSuAshhAgpn2/Xusmf29OMEsKFakL+lyTnfoSKwvbxz0kXedEpndRNK9yXl3hocas6pwlw4QkQmwXIv5MQnVFo9xEUQggRdPJ9KTTVmMDn3e9ndU61tUvtgAQjmyr2f85v5TYEZ4u8rT6f7iseAGDX4FuoTx2ncyIhjk7PGAPpEQrFjSrLSzyc1E1WiTikLqOhJl/vFEKIoyCFuxBCiKCRX+thkutZmAfQdMjn3vxDy2FsMRoAmzl4WuYVr5Nei/+M0dNEXfIYdg36s96RhDhqiqIwqauJd7doy8JJ4d6GjFGw8SPttk+GFwjRmUjhLoQQImg0ubSZpaf3aSI9KX6/n/93rZNdjSpndjcxLv0wvgIby7FtfIfUiIf8HVUfqkr3lQ8QUb0ZtzWO3OOfA0Nw9ioQoaO1cC/yoKpqSA+BaVN8L7CEg6sJyjZCl5F6JxJCHCYp3IUQQgSd9DAvWTH7TuNS0uhjV6OKQYEze5qJth7Gwb3iBaW6nVJ2vC4bZ5CU9xmqYmT78c/hCkvRO5IQx2xsmgmLEXY1qOTV+ugZKyejDspghKT+sGsVFK6AkdfqnUgIcZhkcjohhBAhYfEurVvo4ETD4RXtQSYhbxYZ658HYMfoR6hLO0HfQEL4SZhZ4bhUrViX2eUPQ9IA7bpwmb45hBBHRAp3IYQQQc+nqizdpU1Ed0KX0OtsFlW6jB4r7gOgeMCNlPe+VOdEQvjXpK6yLNxhS+yjXdcWQvUOfbMIIQ6bFO5CCCGCXk61j4oWFbsJRqaEVjdae+12+vx0Mwafm8puZ1A47C96RxLC7yZ11SalW1XmpcEly50dktn+6+2c7/XLIYQ4IlK4CyGECD7qvgfue7vJj041YjGGTjd5e002/X+8ApO7gfrEEeSOfxoU+eoXwScz2kBWtAG3D5bsklb3w5YzV+8EQojDFHr9BYUQQnRermYoXQ9VuVBT8OuluRLcLfRzNJBjdWLe5sOba8drCqfJGM2q6vsBG+ealhNbEUZLWBccYSlBXcSGV22k37xrMDtraIrtS/aJr6AarXrHEqLdnNzNxOsbXPy408Np3WVZuMNSsAScDWCN1DuJEKINUrgLIYQIXPWlsGOBNgPyrl9g92ZQvQd9uhHY26Bu8jRj8jQzz5tFg2ojlSrOKfkPhlKtNd5jCqc+pj/1sQOojx1IU2RW0CyNFln+C33n/wGTu5GG+CFsO/l/eKwxescSol3tLdwXFHrw+lSMhtDpXXNUotKhvhh2LIR+Z+mdRgjRhoAu3P/xj3/w8MMP7/NYnz592LZtGwAOh4O77rqLDz/8EKfTydSpU/nvf/9LcnKyHnGF6PTK6hxsLqlje3kj23c3klveQGWjC1VVUdF6H5uMCt3iw8hKCMdu1oqcFtfBC6mjlVve6LdthVtNZCWE+217oh35vFC8GnK+g+3fQ9mG/Z8TkQLJAyA2c8+lm/aYJYyc3O1cObuJewY1kZUYgcnTxP82xEAtnBRfTXXE8dibd2FrKsbkaSKuchVxlasA8JgiqEkcSVXScdTFD8PXSVuno0qX0nfBDRi9LdQnjWbbpNfwWqQ1TQS/kclGoixQ7VBZW+5lZEpAH+bqr+s42PQJZM+Vwl2ITiDgP9EGDBjAjz/+2HrfZPo18h133MGcOXP45JNPiI6O5pZbbuH8889n6dKlekQVolMqrGrm202lfLOpjPVFtYf1ml01LSzNrWq9/69vttA3JYphXWMY3jWW1GgbinJ0LR02s9Z1+faP1h3V6w9mwd0nSvEeqFQVStfBxk9h02fQUPqbHyqQNgy6jYMuo6DLSK2V6CD7l6ushd1U4DCZcISnUN3i45daBwAjBg9ie8QQ7Yk+L+ENO4iq2UxUzSaiardg8jSSWLqQxNKFeA0WahOGUxk1iBp8dIqprlSV1K0z6brmSQyqh9q0CWRPfAmfyd72a4UIAmajwqSuJr7M9fDjTo8U7m3pNlYr3HPmaidNg6THkRDBKuA/0UwmEykpKfs9XldXx8yZM3n//fc56aSTAPjf//5Hv379WLFiBccdd1xHRxWi0/D6VL5eX8LMJflsLK5rfVxRoE9yJL2SI+mVFEGvpAhSom0YDQoKCooCDreXgqpm8isbWVtYy7K8KnwqbCmtZ0tpPe+tLKRbXBinDkxhXI8ELKYjG0OcGm3n2YuH4HD7/PK7Fte28OKCXJqcMllRwGkogzXvwIYPtTHre1mjoefJ0GsK9JwMEYlH/RY/FXlRgb5xBlIjfrMvGow0RfeiKboXpZnnguolsnYbceUriCtfgc1RQXz5CuLLV+A2GKnaNIOKAX+gMX7wQU8a6MnorKPnsr8Qt0s70V2ZeRa5456UMe0i5JzczawV7gUe7h2jd5oAlzoEbNHaHCFFP2uFvBAiYAV84b59+3bS0tKw2WyMHTuWxx9/nK5du7J69WrcbjeTJ09ufW7fvn3p2rUry5cvP2Th7nQ6cTqdrffr6+vb9XcQIpDM31bOnz9Yy47KJgAMCoztEc9pA1OZMiCZpEhbm9sYmRkHwKbiOs6csYS7TulNZaOTNYW1bCmtZ2d1M68s2sEHPxcyuV8yk/snExtmOeyMqdHSQhi0fD7IXwi/vAHbvvl1vLrJBn1Og0EXa0W76dgLTp+qsrBIO2FzYtc2vu4UIw2xA2iIHcDO3n8grCGfhN2LSSyej8VdR0rh16QUfk1LVHcqup9HRfdzcYWnH3NGf4ioXE+vRbdgayrGZ7BQMPIBdve+PCBPMAjR3iZmmDAZILfWR0Gdj8zo4J2A8pgZTND7VNjwEWybLYW7EAEuoAv3MWPG8Oabb9KnTx9KS0t5+OGHOeGEE9i0aRNlZWVYLBZiYmL2eU1ycjJlZWWH3O7jjz++39h5IYLdlhKtZf3ZH3IAiAkzc/0J3blkVAbxEcdWJMVHWBmZGcepA1NpdHhYkF3Od5vLqGpyMWttMV9vKOGMQWmcMzQNm1m64oUkVxOsex9W/Beqd/z6eMZxMOJqbXyln2c13lblo7xZW7t9TOoR7HeKQnNUdwqjulOYeBLRK54kMaMXcbuXYa/fQdd1z5Cx7lnqU46jovv5VHU9FZ+544dhmBzVdNkwg+Sc9zCoHhwRXcmZ8AJN8QM7PIsQgSLaqjA6xciyEi8/7nTzx8HS6+SQ+p75a+E+5VE54SdEAAvowv20005rvT148GDGjBlDt27d+Pjjj7Hbj75F7r777uPOO+9svV9fX09GRsYxZRUiUNU73Ly5rIDledqY9CibiZtP7MmVY7sRYfX/R0CEzcRZQ9I4fVAqqwqq+WZjKdvLG/liXTELs8u5eFQGE3slYpDZfkNDQxn8/KrWwt5Soz1mjYYh02DEtZDcv93eekGh1to+Ns2IzXSU+5tioI4I6obeiyEqhfid35K443Oid68gumw50WXLyVr5d6q7nUpF9/OoSx7b7uNEFa+TlG1v02XjC5jcDQBUdjudHcc9htcS1a7vLURncHI3E8tKvMzb6ZHCvS09T9Z6PNUUQPkWbeJPIURACujC/fdiYmLo3bs3ubm5nHLKKbhcLmpra/dpdd+9e/cBx8T/ltVqxWqVD3IR/FbmV/HGknzqHR4MCvhUmHn1KEZlxbX7exsNCsd1j2dMVhy/FNTw3s872V3v5NVFO/h+cxnXn9Cd7okR7Z4jpFXlaevz6qFxN6x9F7K/BZ9beywqHQZdpHXNNNvB64KSdf5939pCwE6LB1aWat3wJ7XVTf4w+cwRVPS8iIqeF2FpLCYx/wsS8z7D3lBA4o7PSdzxOS57EtVdTqYmYwp1Kcf5dYy5uaWChPwvScl+B1tjEQBNsf0oGHE/9anj/fY+QnR2k7uZ+edyJz+XeqlzqkRb5UTxQVnCofskyPkWts2Rwl2IANapCvfGxkby8vK48sorGTFiBGazmXnz5nHBBRcAkJ2dTWFhIWPHyhgdEdpcHh+vLd7BktxKALrE2jl7SBr/XZiH3dKxXdUVRWFUVhxDu8bw3eYyPl9bTEFVM3//cjPnDU/nnKFpmAwyBtHvqvJgxnC9U+yrvhiWPq9d2osvE3iM9fU23D7oEqnQI8b/+5crIp3iQdMpHvgnIirXkbhjFgkFs7G0lJOy/QNStn+AxxxBXeoJ1CeNoiFpOM2x/VAN5iN6H6OrnpiSxSTu+IyYksUoe+YEcNmTKBx6FxXdz5eZoIX4nW7RBnrFGthe42NhkYdzeh7Z313I6XvGnsJ9Nkz8q95phBAHEdCF+913381ZZ51Ft27dKCkp4aGHHsJoNHLppZcSHR3Nddddx5133klcXBxRUVHceuutjB07VmaUFyGtusnFM99ns6OyCYMCZw9J5/zh6eyqadE1l9lo4MzBaUzolcgbS/NZmV/Np6t3sbawhptP7El6jExI51d7W9pPuAuiO2AokKNOW1KoaCWoe1YESOgNvaZCfI/2f/+9akwwD1ZXWQGVSRmmo16a8LAoCo2Jw2hMHEbByAeI2r2SuKLviSuah6VlN/GF3xJf+C0AXqONpvhBOCK74QxPxRWWiissBdVgQvE6MXhdGLxOrE3FhFdvJrx6C7bGwn3eriFhGBU9zqci6zx85rD2+72E6ORO7mZie42LHwvcUri3pc9poBigdL3Waymmq96JhBAHENCF+65du7j00kupqqoiMTGR448/nhUrVpCYqC0N9Nxzz2EwGLjgggtwOp1MnTqV//73vzqnFkI/eRWNPPN9NjXNbiKsJu44pTf9U/cd85pb3ui39zuabUXZzdx2ci+W5VXxv6X55FU0cd+sDVw1NpOT+ya1b5EViqIzIL5n+23f1aitvb7lK/DuWa0jbRgMuRSS2m/8+kH5vEATxY0qRgWO79JxX3Oq0Upd2gTq0iaQP/oRIqo2El2ymMjKNURWrMXkqiOqfBVR5auOaLuOyG5UdjuDiu7n44ju3k7phQgup3Qz8fI6FwuLPLi9KmajfLccVHgCdB0LO5dqq30cd5PeiYQQBxDQhfuHH354yJ/bbDZefPFFXnzxxQ5KJETgWp5Xxcs/5eHy+kiPsfOXqX1Ijvp1aTebWesufPtH6/z+3nu3fbgURWF8zwT6pUbxyk95bCiuY+aSfPLKG7l2fNYRr/0udODzaOMhN3z4a+t+Yj8YcU3AjJEcmWIkSq+xrYqBxoQhNCYM0e6rPmz1+URUbcDaVIKlqQRrcxmW5jJARTVY8Bmt+IxW3NY4muIH0hTbn+a4/nisMfr8DkJ0YkOTjMTZFKodKqvKvIxLD+hDXv31PWNP4T5bCnchApR8igkRBOZt283ri/MBGJoRw60n9STMsu+fd2q0nWcvHoLD7fPre9vMhqNedz0u3MK9p/Xl6/UlfPhLEQtzKiiqaeaOyb2PeYk60Y6K18CqV6Ful3Y/uisMvxoyRuu+lJDDo7bebnPt9o6kGHBE98AR3YHDBoQIYUaDwkldTXya4+b7Ao8U7m3pewZ8dz/sXAbN1RDW/pPYCiGOjHyKCdHJ/bZon9I/mavHZh50qbWjLbDbk6IonD00ncyEcGbMzyWvoon7P9/IbZP37+YvdNZQBqteh6IV2n1bNAy7EnqeEjATpP1UpC0BF2dTGJwYwD036ovB7ed5J8x2beZ+IQQAU7L2Fu5uHhpnlaFYhxKbCcmDYPdGyPkOhl6qdyIhxO9I4S5EJ/bbov20gSlceVy3TntgMrhLDP86dyDP/pDDzupmHvtmKzdP7MH4ngl6RxM+D2yaBes/0JZ2UwzQ7yxtHLslcJb0U1WVOTu0wn1MqhFDoP4t1BfDrBvaZ9vnvyrFuxB7TOhiwm6CkkaVjZU+BicGxgnGgNX3DK1w3zZbCnchApAU7kJ0UsFUtO+VFGXj4XMG8NLCPFbmV/PCglyqm1ycOTi10/9unVZFNiyfATUF2v3UITD6xoCcdXjNbi876rShICOSA/gAfW9L++CLITzJP9tsKocNH/u/FV+ITsxmUpjU1cQ3Ozx8l++Wwr0tfc+An56A3HngatLWeBdCBAwp3IXohJbkVrYW7acPTOGKICja97KajPz55F68u2In324q4/2fC6lqcnHVcd0OOgRAtAO3A9a+DVu/BlSwRsGo66H7ibqPYz+Ydza7W2+HmQMz4z7CkyBaWseFaE9Ts8x8s8PD3HwPfxmtd5oAlzJI6zJfU6B1lx94vt6JhBC/EcADAIUQB7KltJ6Xf8oD4NQgK9r3MigKV43N5MrjugHw3eYy/jNvO26vfyfWEwdRvhW+vhW2fgWo0H0SnPsS9JgUsEV7ZYuPb3a4236iECKkTMowYTZAXq2P3Bqv3nECm6LAgPO025s/1zeLEGI/UrgL0YkU17bw7A/ZeH0qx3WPC4ru8Ydy+qBU/nxST0wGhZ8Lqnn6+2ycHjnwajdeN6x5B+beAw2lEJ4Ikx+BE+7SJqILYB9tc+PyQe9Y+VoTQvwqyqowfs+M8nPzPTqn6QT2Fu7bfwBno75ZhBD7kCMcITqJuhY3T87dRpPTS6+kCG6e2DNwJ9/yo7E9Erjn1L5YTQY27KrjybnZONxSvPtdbRF8czds/AhUn9bKfvYMSB+ud7I2eX0q729xAXBGdxkBJoTY16lZewt36ZXTppTBENcdPC2w/Tu90wghfkMKdyE6AZfHxzPfZ1Pe4CQp0srdU/pgMYXOn+/A9GjuPa0vdrORLaX1PP7tVppd0nLiN3nzYM7tUJ0H1kiYeK/Wyh5AM8YfyvxCD8WNKjFWhRO6SOEuhNjX5EwTBgU2VfrY1SBDrg5JUaD/udpt6S4vREAJnSN/ITopVVV5dVEe28sbCbcauefUvkTZzXrH6nB9U6L42xn9CLcaydndyKNzttLokOL9mHgcsPQ/sOQ58Di1GePPfhEyj9c72RF5Z7PW2j6trxmLMfh7oQghjkyC3cDIFG1G+e+k1b1t+3SXb9A3ixCilTRNCBHgVuZXszSvCoMCd57Sh7QYu96RdNMjMYIHz+jPv77ZSn5lE499u5W/nd6PcKt8lB2x2iL46XGoLQQUGHoZDLoYDJ1ruaTcGi+LdnlRgMv7W6h3qnpHEkK0o9zao2sxH5Jo5OdSL7Ny3IxJ/fU7I9wCWdGd63Ov3aUMgrgeWi+snO9g0IV6JxJCIIW7EAFvzsZSAC4b3Y3+qVE6p9Fft/hwHjyjP4/O2UJ+ZRP/nruNe0/rS5hFPs4OW+FyWPysNobRFgMT/qK1tndCr2/QWtunZJroGmVgU4XMfyBEMLLt+Yi/fX7LMW1nc5WPM2c17fPYgkvCpXj/rb2zyy9+WusuL4W7EAFBjnSFCFANDq07n9enMiozltMHpeicKHBkxIVx/+n9eHTOVraXN/Lk3GzuPa0vNrMceB2S6oP1H8L697X7KYNgwl/BHqtvrqNU3uxjVo72d3L9EIvOaYQQ7Sk1wsCzk2wcywipF9c6KW5UOa+niVGpJoobfby41kWTy385g8bewn1vd3lrpN6JhAh5UrgLEYB8PpVnf8gBIC7cwk0TewT1sm9Ho1t8OPef3o9/zdlC9u4Gnvxu257Z56V4PyB3s9bKXrRCu9/vLBh5HRg679fAO5tduHwwLMnIiGT5fxci2KVGHNvUTMd3MfHRNjd5dT4u7ifTPB1S8gCI7wlVuZA9FwZfpHciIUJe5z1iEyKIvfRTHqsKagC4bHRX6QZ+EFkJ4dx3ej/+NWcrW0sbePb7HO6e2gezUQ7I9tG4G+Y9rI1nN5jguOnQ6xS9Ux2TZrfKO5u11vYbhljkxNZedUWBtR0hAsiYVCMfbXOzqdIn82G0ZW93+UVPad3lpXAXQndSDQgRYNYV1ba2tgMhPRnd4eiRGMG9p/XlsW+2sqG4jhcX5PLnk3phMEghB2jF+i9vgKMW7HEw6X5I7Kt3qmP2WY6bWqdK1yiFKZnyVYbJql0verp9titEEEiNMNAtSmFnvcqqMi/do+Uk7yHtLdxzfwBHPdhknh0h9CRHO0IEkBaXlzs/WofXpzKhVwKLtlfqHalT6J0cyZ2n9ObJ77JZmV/NzKX5/PH4LGmFBVj+AnhdEJsJJ/8DwhP0TnTMvD6V1zc4AbhukBWjnKTR/l9PuEtb1s9fTNag2F+E+K1x6SZ21rtZXuKhe7TMjXFISf0hoTdU5sC22drqI0II3UjhLkQAefzbreyobCI5ysrNJ/aQwv0IDO4Sw62TevKf+duZv62cCKuJS0d31TuWfjZ9pl17XZA2HCbeC5YwfTP5yQ8FHnbWq0Rb4aI+Zr3jBA4psoVo03FpRj7Y6mZLpY8Gl3SXPyRF0ZYJXfCoNrGpFO5C6Er6CAkRIH7KqeDt5TsBePqiIUTapCA5UmO6x3P98d0B+Gp9CV+tK9Y5kQ5UFb5/EJb9n3a/61g4+e9BU7QDvLZnCbgr+1sIM0truxDi8CWFGegRY0AFNlXK8pFt2ju2PX8R1Jfom0WIECeFuxABoLbZxV8+WQ/A1WO7cUKvRJ0TdV6T+iZx+Ritpf2DVUWsyq/WOVEH8nrgq1t+LdpBay3pxDPH/96qUg+rd3uxGOCqgdLNVQhx5MalaatQbKiQwr1NsZnaCWBU2Pip3mmECGlSuAsRAB74YhPlDU66J4Zz72n99I7T6Z05OI1zh6YB8EWotLq7HfDJ1bD2XVAM2vrsoHV1DCL/Wa2N4b6gt5mkMPkKE0IcuTF7Cved9dJV/rAMvli73vCRvjmECHFy1COEzuZuKmX2hlKMBoXnLh6K3SLrUfvDxSMzmNwvib2HZat31uiap1056uG9C7XJg4xWuPgd6HuG3qn8bnWZhyXFXkwG+NNwme1cCHF04u0G+sTJIfBhG3AeGC2wexOUbdI7jRAhSz61hNBRXYubB7/cDMDNE3swJCNG30BBRFEUrh2XxeAu0QA89s1WfikIwm7zzdXw9tlQsBgskXDFZ9DvTL1TtYvn97S2X9jbTEakfH0JIY7e2DQ5SX7Y7LHQa4p2e+PH+mYRIoTJkY8QOnri261U7Okif8tJPfWOE3QMBoULR3QBwOnxce2bq9haWq9zKj9qqoS3zoaStRAWD9fMhqwT9E7VLlaXeVi8S2ttnz5MWtuFEMdmTKqJvQOJypp8umbpFAZP0643fAI+mRtACD1I4S6ETpbnVfHBz0UAPHH+YGxmOfvfHkwG7WOuf2oUDQ4PV73xM4VVzTqn8oOG3fDmGbB7I4QnwTVzIG2o3qnazd6x7ef3MpMRJV9dQohjE2NTyIrWPkuW7PLonKYT6D0VbNHQUAIFS/ROI0RIkqMfIXTgcHu5//ONAFw+piujs+J0ThT8/n5Wf/qmRFLR4OSKmSspb3DoHeno1RXDm6dDxTaITINrv4Wk4J3UcO1uD4t2eTEqcIuMbRdC+MngRO0weNEuaUFuk8mqjXUHmaROCJ0EzxpBQugsv7KJJufhnbV/a1kB+ZVNxIVbOHtIGpuK6/Z7Tm55o78jhrQIq4m3/zCaC19eTmF1M1fN/JmPbhxLtN2sd7QjU7dLa2mvKYDornD1VxCXpXeqdvWf1dq67ef1MtNVWtuFEH4yMMHIF7kedtT52F7jpVes9Hw7pMGXwOo3YctXcPrTYAnTO5EQIUUKdyH8IL+yiUlPLzzi11U3uZj26opDPsdmlkLFX5KibLx73RgueHkZ28oa+ONbq3j7D2M6z0z+9SXw5pla0R6bCVd/DTFd9U7VrtaVe1lY5JHWdiGE34WZf10uc1aOm3vGdJLvAr1kjNG+c2oLIfsbGHSh3omECClSuAvhB3tb2qdP6kl6jP2gz/OpKq8u2kFhdTMD0qK4fEy3Q27XZjaQGn3w7Ykj1zU+jLf/MJqLX1nOqoIapr+/hleuHIHZGOAnSOpL9xTt+RDTTRvTHt1F71TtSlVVnlipDWk4t5eZzOgA/z8SQnRaX2x3c/coK0aD0vaTQ5XBoE1St+gpWPuuFO5CdDAp3IXwo/QYO1kJ4Qf9+U855RRWN2M1Gbh5Yg/iI6QFUQ/9UqN445pRXDlzJfO3lfPXTzfwzEVDMATqAVtDGbx1FlTnaa0d18wO+qIdYH6hhxUlXixGuGOk/K0IIdpHuBlKm1RWlHgZ30UOjQ9p2BVa4b5jwa+9v4QQHUKaL4ToII1OD++vLATgwhFdpGjX2ajMOF66fAQmg8Lna4t5ZPYWVFXVO9b+9i75VrUdojPg6tlB3z0ewONTeWyFNpP8tQMtdJF124UQ7WTCnmL9s+1unZN0ArGZ0P1E7fba9/RMIkTIkSMhITrIx78UUe/wkB5j59SBKXrHEcCkvkk8fdEQAN5cVsAL83N1TvQ7jjp45zyozIaodG1Me+yhh1cEi4+2ucmr9RFrU/iTrNsuhGhHJ3XVCve5O9w0uQPwBG6gGX6Vdr32XVnTXYgOJP2BhOgAOyoa+XHLbgCuHZ/Zura40N+5w9KpaXbx8NdbeOaHHKLDzFw1NlPvWOBqhvenQdkGCE+Eq4J/9vi9Gl0qz/2itbbfNsJKtDUAhjDUFQXmtoQQx6xvnIHMKAMF9T6+y3dzfm+L3pECW98zwR6rremeOw96T9E7kRAhQQp3IdqZT1V5Y2k+KjCuRzwD0qL1jiR+59rxWdQ0u/m/edv5+5ebCbOYuHCEjmPIPS74+EooXA7WaLjyc0joqV+eDvbqeieVLSqZUQYu66fzcn2mPa39i55uv20LIXSlKArn9zbz7C9OZuVI4d4mkxWGXAor/gtr3pLCXYgOIoW7EO1sYXYFeRVN2M3GNmeRF/q5Y3IvGhxu/re0gL9+up4wi5HTB6V2fBCfF2ZdD7k/gjkMLv8EUgZ1fA6dlDX5eHWDtm77vWOsWIw6t7aHJ8AJd4HH6d/tmqzatoUQAeG8XlrhvrTYS2mjj9QI6Rl3SMOu1Ar3nLnQsBsik/VOJETQk8JdiHbU5PTw0SptQroLhnchLlzO4uslt7yxzeecPyyd4poWvt+ym1s/WMvuOgejsuIO+Nxwq+mQKwgcsao8cNRrs/VmzwGDGU55RCvwStYd+fYqc/yXrQM99bMThwdGphiZmhUgX1FSYAsR9DKiDIxOMfJzmZcvct3cPFR6xBxScn/oMgp2rYL1H8Dxt+udSIigFyBHRUIEp1lri6l3eEiLsTF1oJyN1oPNrLWa3P7RuiN6nden8vDsLYd8zoK7T/RP8V6VBzOG7/uYzw3f3H3s2zbbj30bHWR5iYfPcrRZnf92nBVFCYCx7UKIkHF+bzM/l3mZlePmpiEW+Qxqy/CrtcJ9zdsw/jaQfy8h2pUU7kK0k5LaFr7bVAbAlcfJhHR6SY228+zFQ3C4fYf9Gq9P5f2fC9laWo/ZqHDNuKx9CvTi2hZeXJBLk9Pjn5DOhn3vD7kUMsYc+3bNdm02+k7A6VX522IHAJf3NzMsWb6ehBAd6/TuZv6+1MH2Gh8bK30MTjTqHSmwDTgP5t4L1XmwcylkHq93IiGCmhwZCdFO3lmxE6+qMiwjhqEZMXrHCWmp0Ufe6nzfaX15+vtsNuyq4+3lBdx7al/6pka1Qzpg06xfb4+6Hvqf0z7vE8BeXudiR62PBLvCX0fb9I4jhAhBUVaF07JMfJnr4YOtLgYndp4eS7qwRsDAC7QJ6la/JYW7EO1MmgCFaAfrimpYV1SLUVG48jiZkK4zMhsN3HVKHwanR+P0+Hhi7ja2ldX7/43WfwTL/qPd7n1qSBbtO2q9vLhGm/ztoXG2wFj+TQgRki7tp81F81Wum0aXrOnephHXaNdbvoDGcj2TCBH0pHAXws88Ph/vLN8JwKkDU0iNkTP2nZXFZOCuKX0YuKd4//fcbeTsbmj7hYcrbz58+adf7/ea6r9tdxKqqnWRd/lgYoaRM3tIRzAhhH7GpBrpHm2gyQ1f57n1jhP40odD+kjwurRWdyFEu5HCXQg/+37zbkrqHETZTJw/vHOMLxYHZzEZuHtKbwakReFw+3j8263kVzYd+4bLNsJHV4HPAz0na4+F4MQ+s7a7WV7ixWqEfx5vl8mghBC6UhSFS/uZAfhgq0vnNJ3E6Bu0619mgldOdgjRXqRwF8KPmp0eZq3ZBcDFozIIs0jrYTCwmoz8ZWqf1uL9zWX5x7bBul3w3kXgaoDME2DiPf4J2snsbvLx6HKti/xtI6x0jZKvJCGE/i7obcZigA0VPjZVePWOE/gGnAvhidBQClu/1juNEEFLjpKE8KMF2eU0ubx0jQtjUu8kveMIP7KajPx1al+GZsTg9mrjHlfuqDryDbXUwrsXagc4if1g2rtgtPg3bCfgU1XuWtBCjUOlf7yB6weH3r+BECIwxdkNTM3STrx/sE1a3dtkssKIa7XbP7+mbxYhgpgU7kL40Yod1QBcPqYrBoN0+Q02FpOBO0/Rus0D/OubrXy9vuTwN+BxwkdXQMVWiEyFKz4Fe0z7hA1wMze4WFLsxWaC/zvZjtkofy9CiMCxd5K6L7e7aXLLJHVtGvkHMJigcJk2FEwI4XdSuAvhR15VZUiXaAZ3idE7imgnZqOBS0Z1BcCnwm0fruX9lYVtv9Dngy9uhoLFYImEyz+B6C7tnDYwbar08uTPWhf5B8fa6BkrayULIQLL2DQjmVEGGt0wWyapa1tUKvQ7S7u98hV9swgRpKRwF8IPtpRoy4QpwOVjZPm3YGfc05vi1AEp+FS4//ONPPtDDqp6iFaZef+ATZ9pLRLT3oGUQR0TNsC0uFVum9eC2wenZJq4bM8kUEIIEUh+O0nd+zJJ3eEZfaN2vfETaK7WN4sQQUgKdyGOkaqqzFyiTVY2MjOWjLgwnROJjjJ9Ug/+fHIvAP5v3nbum7URj9e3/xN/fg2W7lmr/ewXoMekDkwZWP653EFerY+kMIV/T7DJLPJCiIB1QR8zZgOsL/exuVImqWtT1+O0k9IeB6x9R+80QgQdmfJaiGM0Z2Mp2XvW9j65X7LOacQB1ReDu8V/26v1AKBU5nDnAAvJvhgeXFjLh6uKqKysYMapsdjNe86LFiyG7x/Ubo/8IyT1g5J1+26vMsd/2QLY5zku3t+qdTl9dpKdOLucOxZCBK4Eu4EpmSbm7PDw7mYXj0+06x3Jv/z53WONhPgeWqv7V7fAz6/DcdPBKKWGEP4if01CHAOnx8u/525rvR9lk26/Aae+GGbd4N9tqinA9TDrejAUcDmQYBrJn9238GM+THt5Ca9aniVFqdn3db+8rl0OxhxkB4W/sbrMwz0/OQCYPszC8V3k60cIEfiuHmhhzg4Ps7a7+esYK7G2IDjhuPe7Ztb1/t3urWtg0IXw40NQVwhbvtDuCyH8Qo6chDgGby/bSVF1C3HhFqqbZAxcQNrb0j74Ygj30xJ9DUZYC5xwN8Rqre9Tgfcqm7h+mYkNrh6czX941fg0Q70btGXfRv0RDIeYhM1sh6h0/+QLMMUNPm78vgWXD6ZkmrhrlFXvSEIIcVhGpRgZmGBgU6WP97e6mT4sCD6/otLhvFf81xOtrggWPwPOBu27bPSNsPAxbYjYwAtAhkQJ4RdSuAtxlGqaXMyYvx2AK4/rxn/mbdc5kTik8CSI9lNhrPoAB0RnQPyvxfjIePgq1cd13zaRU2viYuedPBX9GedMvhIsoTn3QZNb5Y/fNVPZotIv3sBzJ9kxyEGcEKKTUBSFawdZuGuBg7c3ubhhsCU4lq9szxPFo6+Hpc9D2QbYsTCk53URwp+CoL+PEPqYMT+XeoeHvimRnNTXTy25otPLiFT4LPE1JhtW48LCbXWX8vgaAx5f6K0D7FNV7pjfwtYqHwl2hdenhhFuDoIDXiFESDmzh5kEu8LuZpVv8j16xwl8YXEw/Crt9t6JWYUQx0wKdyGOQkFlE++sKADgb2f0a10eTAg2fEhk0XxesfyHm3vVAfDKeheXzW5md9MBZpwPUqqq8vclDr4v8GAxwitT7KRHyleOEKLzsRoVrhxgAeCNDc5DL/0pNMf9CRQj7Fiw/4SsQoijIkdRQhyFf8/dhturcmKfRE7olah3HBEoCpbAuvcAMI69mXtOyuCFyXYizPBzqZfTP21i8a7gb63ZW7S/u8WNAjx1op0RKTIySwjReV3e34zFAOsrfKzZLUvDtSm2Gww8X7u97P/0zSJEkJDCXYgj9EtBNd9uKsOgwH2n9dM7jggUVbmw5Dntdr9zoPdUQOti+fUF4fSLN1DlULlqTjPPrHLg9gZni42qqjy01ME7rUW7jXN6ymoLQojOLcFu4Jxe2mfZGxtlMtrDMu7P2vXmz6GmQNcoQgQDKdyFOAKqqvLonK0ATBuVQZ+USJ0TiYDQXA3z/wleJ6SPgJF/2OfHWdFGPj83nEv7mVGBGWtcnP9FEznVwdVqo6oq/1jq4O3NWtH+5Ik2Luxj0TuWEEL4xbWDtM+zufkeShpDZ+jTUUsdDD1O0iZ0Xf6i3mmE6PSkcBfiCMzeUMq6olrCLEbumNxb7zgiEHicsOBRaK6C6C4w4a8HXPbNZlJ4fIKd/zvZTrQVNlb6OPOzJl5e58QbBBPXubwq9y9y8Naeov3fE21cJEW7ECKI9I83MjbNiFeFtzZJq/thGX+7dr3mHWiq1DWKEJ2dFO5CHCanx8u/524D4MYJPUiKsumcSOhOVbWxe5U5YI2Ek/4OlvBDvuTsnma+vyiCk7qacPngiZVOLvqqmexO3Ppe1eLjitnNfLBNK9qfmGjj4r5StAshgs8f9rS6v7/VRZ2z8590bXdZEyBtGHhaYNkMvdMI0alJ4S7EYXp72U521bSQHGXl+glZescRASAh7zPI/0mbOXfifRCVdlivSw43MPNUO09OtBFhhjW7tYnrHl7qoL6THQhuqfJy9qwmfi7zEmGGmafamSZFuxAiSJ3czUTvWAMNLnh7s7S6t0lRYOI92u2fX5NWdyGOgUzzK8RhqGlyMWP+dgDumtKHMIv86YS6CYb1JOe8q90Zc6M2lu8IKIrCxX0tHN/FxD+XOfg238P/Nrn4Os/NvWOsnN/bjEEJ7GUGv85189efWmjxQGaUgddPtdMzdv9hAoeSX+elyY/Hvrm1Mu5UCNF+DIrC9GFWbpvfwhsbXfxhkIVwc2B/Vuuu96mQOhRK12mt7qc8rHciITolqT6EOAwz5udS7/DQNyWSC4Z30TuO0Fm4s5z/mF9EQYVeU6HP6Ue9rbQIAy9NCWPxLg8PLXWwo9bH3QsdvLHRxR0jrUzuZkIJsAJ+d5OPh5Y6mJuvLW13QhcjL0wOI9p6ZDnz67xM+rCpPSJik283IUQ7OaOHied+MVBQ7+P9LS6uH2LVO1JgUxQ48T74YJrW6j7uVghP0DuVEJ2OHNoI0YaCyibeWVEAwN/O6IfREFhFlOhYBq+TE3L/TazSSHN0L8LG3OSX7Z7QxcTcC8N5c5OL/1vtZEuVj+u/a2FQgoE7RlqZ1FX/At6nqny0zc1jKxw0uMBkgJuGWLh9pBXTUfxd7G1pnz7MQnqE/0Zu2UyQ6sftCSHEb5kMCjcPs3DPTw5e3eDiygEWbCY5Njik3lO1se4la7W5YU55RO9EQnQ6UriLkJRf2UST03NYz33sm624vSojusUSG2ZhU3Hdfs/JLW/0d0QRiFSVrK0vE9ecT6UaRdXwe+hj9N8a5Rajwg1DrFzUx8xr6128ucnFxkoff5jbQv94A1cNsHBOTzP2Du6Wqaoqy0u8PP+Lk5/LtEn0BicaeGKinf7xR9Y1/kDSIwxkxUihLYToPM7rZeY/q52UNKp8ku3mygEyt8ch7W11f//iPa3uf5ZWdyGOkBTuIuTkVzYx6emFR/y61TtrOHPGkkM+x2aW4iOYJe+aS1LpAnwYuNV9K3+zJ7bL+8TaDPx1jI3rBlt4db2Ltza72FLl495FDh5b4eDCPhYu62c+4vHkR8qnqvxQ4OGldS7WlWsFu90Ed42ycu1Ai/Q+EUKELItR4cYhVh5a6uDldU4u6WvGbJTPxEPqNUVa3YU4BlK4i5Czt6V9+qSepMfYD/o8n6ry8k957KppYVRmLOcNO/TYdpvZQGr0wbcnOreI2m1kZs8EYH2XK1ieO6Dd3zPebuC+42zcNNTCJ9lu3t3iorBe5Y2NLt7Y6KJXrIGpmSamZpkZmGDwS1d6VVXJrfXxfb6Hz7e7Wyd7sxphWl8zNw6xkh4pJ6iEEGJaXzMz1jgpblT5fLtblsFsi7S6C3FMpHAXISs9xk5WwsHX3F6SW8mumhZsZgN/GJ9FTJh8IYcqs7OWPhuexKB6qEoax9aUcyDX2WHvH2szcMMQK38cbGFRkZd3t7hYWORhe42P7TUuXljrIjVcYXiykSFJRoYkGhmYaDysmY69PpWiBpWcGi+ry7z8UOBhR92vM7NHWuDK/hauHWQhMUwKdiGE2MtmUrh+sIXHVzp5aZ2L83ubj2q+j5DSawqkDYeSNbD4WTj1Mb0TCdFpBHTh/vjjjzNr1iy2bduG3W5n3Lhx/Pvf/6ZPnz6tzznxxBP56aef9nndjTfeyMsvv9zRcUUQcXq8fPBzIQDnDE2Xoj2EKT4PvTc8hcVZTXN4F3IH3AqN2oGZP5ceC7dAVvShu74bFIUTu5o4sauJOqfKgkIP3+W7WVjkobRJZc4OD3N2aD1KFCDerpBgV0gMU0gMM2BSwOUFp1fF6YXdzT5ya3w4vfu+j8UA47uYmJJp4ozuZqKOcLZ4IYQIFZf3t/Dyehf5dT4+zXZzST85XjgkRYGTHoB3z4dVr2nLqcZ20zuVEJ1CQBfuP/30E9OnT2fUqFF4PB7uv/9+pkyZwpYtWwgP/7Wl9Prrr+eRR34dJxMWFqZHXBFEZm8opbrJRUKEhdMHpuodR+io6/a3iardjMdoJ3vIffhMdmwmrWC/fX6LX99rwSXhbRbve0VbFc7tZebcXmYcHpXVZV7WV3jZUOFlfbmX0iaVyhbtsq0awHvQbVmN0CPGQN94I5MytBMDkRYp1oUQoi0RFoXpwyw8utzJc784dZlAtNPpeTJ0PxF2LIT5j8IFr+mdSIhOIaAL97lz5+5z/8033yQpKYnVq1czYcKE1sfDwsJISUnp6HgiSFU3ufh6fQkAl43uhsUk3YNDVXzZYtIKvwIgd+BtOMLTAW2psWcn2XAc3sIEbSpu9PHiWlfr8mhHymZSGN/FxPguv36kV7X42N2sUtGsUt7so6JZRUUr0q1GBasRYmwKvWONZEQqMtGcEEIcpSv6W/jfRhfFjSpvbnZx81BZ171Nkx+GVyfCxo9h3C2QOkTvREIEvIAu3H+vrk5bhisuLm6fx9977z3effddUlJSOOuss3jwwQcP2erudDpxOn8dn1pfX98+gUWn9OGqQpweH32SIzmue1zbLxBByd5YRI/NLwCwK/MCapKO2+fngb5OeLzdQLwdiNc7iRBCBDebSeGOkVbuXujgpbVOLutnIVqGGB1a2lAYeCFs+hR+eAiu+kLvREIEvMA+8vwNn8/H7bffzvjx4xk4cGDr45dddhnvvvsuCxYs4L777uOdd97hiiuuOOS2Hn/8caKjo1svGRkZ7R1fdBJ5FY0s3l4JwJVju/lllm7R+Ri8DnpveBKjz0lt3BCKel6mdyQhhBAB7LxeZvrEGah3wX/XdtzkpZ3ayQ+CwQw7FkDefL3TCBHwOk3hPn36dDZt2sSHH364z+M33HADU6dOZdCgQVx++eW8/fbbfP755+Tl5R10W/fddx91dXWtl6KiovaOLzoBVVV5Z/lOAE7omUCPxAidEwldqCpZW18hrKkIlzWO7YPuAKV910sXQgjRuRkNCn8ZpXWRf3OTi9JG/01eGrRiM2HUH7XbPzwEPvk3E+JQOkVX+VtuuYXZs2ezaNEiunQ59FraY8aMASA3N5cePXoc8DlWqxWrVcYfiX2tzK8me3cDFqOBaaOkF0aoSiyZR1LpAlQM5Ay6C48lRu9IRy2/znvU4+YP5nBmvxdCiFB0cjcTo1KMrCrz8p/VTp6YaNc7UseqzDny1/Q5Fda8DWUbYMkz0POUX39mjYT4Ax/LCxGKArpwV1WVW2+9lc8//5yFCxeSlZXV5mvWrVsHQGqqzAQuDp/L4+P9ldryb2cNSSU+Qk7shKKwhgK6b3sVgMKel9EQO0DnREcvv87LpA+b2mXbRzL7vRBChApFUbh3jJULvmzm42w3fxhkoXdcCHxWmvecoJh1/bFtZ/6j2uW3bl0jxbsQewR04T59+nTef/99vvzySyIjIykrKwMgOjoau91OXl4e77//Pqeffjrx8fFs2LCBO+64gwkTJjB48GCd04vO5NtNpVQ0OokLt3Dm4DS94wgdGDwt9N7wFAafi5r44ZRknq93pGOyt6V9+jAL6X6aSO9YZ78XQohgNyLFxJRME98XeHhoqYP3zwwL/vlyotLhvFfAfZRLpHpdsPBxaKmBXlOhz2lQVwSLnwFng3+zCtGJBXTh/tJLLwFw4okn7vP4//73P6655hosFgs//vgjzz//PE1NTWRkZHDBBRfwwAMP6JBWdFa1zS6+XKct/3bJqAxs5hA4Oy72pap03/oS9uZinNZ4cgfeDkrHTwGSW+u/8X17t5UeYSArxr+/i79y+vP3FUKIQPHgWBs/FTWyvMTL7DwPZ/U06x2p/UWlH9vrR98IPz2hTVQ3eBpEy5BFIX4voAt3VVUP+fOMjAx++umnDkojgtXHv+yixe2lR2I443sm6B1H6CCp+HsSyxahKga2D74bjyWqQ9/ftueT+Pb5R9lacRjb9ue2/J3TnxmFEEJvGVEG/jTMynO/OPnXCgcndTMRbg7yVvdj1W08pAzWxrr/MhMGXaR3IiECjhwuiZBWUNXEwuxyAK48LhNDsHdnE/sJa9hBVvbrABT2vIKGmH4dniE1wsCzk2w4PP7drs3k3/Xm2yOnvzMKIUQguHGIhc9yXBTWq/zfaif3HWfTO1JgUxQYfQN8/WfYuRRSh+idSIiAI4W7CFmqqvLuikJU4LjucfRJidQ7kuhgRk8zvdc/hcHnpiZhJCXdztUtS2cpXjtLTiGE0JPNpPDQOBvXzW1h5kYXF/Ux0zNWhuIdUmwm9Dkdts2GzZ/rnUaIgCNHYCJkbS6pZ0tpPWajwmWju+odR3Q0VaX7lv9ibynFaUsgd8CfdRnXLoQQIjid3M3MyV1NeHzwj6WONoeACmDo5WCNgoZSvZMIEXDkKFWErDkbtS+FswankRgpXdhCTfKuuSTsXoJPMZIz6C8dPq5dCCFE8HtovA2LEZYUaxPViTZYI2HYFb/eb67WL4sQAUa6youQVdfiJiHCwtlDZfm3UBNen0dm9kwACntdRWNMH50TCSGE0Iu/V7gIt0BWtNYtvmuUgel7Jqp7aKmDselGEuzSbnZIvabC1q+gbhcsfwF6nqR3IiECghTuIuQU1/46I/ZVYzOxmmTMWSgxupvoveFJDKqH6sTRlHY9W+9IQgghdNCeK3osuCS8tXi/eaiFb3e42Vbt48HFDv57ij3413Y/FgYjDJoGS56BvHmw/UfoNVnvVELoTgp3EVJUVeXVn3YA0Ds5gpHdYnVOJDqUqtJjywvYWnbjsCXtGdcuB09CCBGK2mOljOJGHy+uddHk+vUxi1Hh6Ul2zv28iW/zPaGztvuxiPnNOu5z7oQ/rQBLmH55hAgAUriLkPL9lt2sLqwB4MzBaXLGO8SkFM0hvnw5PsVEzuC/4DVH6B1JCCGEjjpqpYyBCUamD7Pwn9Uu/r7UwXFpRhLDpMt8m8KToHYn/PRvOOVhvdMIoSv5xBAho8Xl5ZGvt7TeT4iw6phGdLTwxp10y3kTgJ29r6Epupe+gYQQQoSU6cOs9I83UONQeWCxzDJ/WMbfrl0vmwFlm3SNIoTepHAXIeOFBdsprm0hUQr2kGPES+/cmRhUD1VJx1GWcYbekYQQQoSYvV3mTQb4rsDDV7kyy3ybMsdDv7NA9cLXt4HPq3ciIXQjhbsICdllDbyyZ2z79ROydE4jOpTqo6ehBJurCoc9mbz+t8i4diGEELroH2/k1uFaA8IDS1rYWeffGe2D0mlPgiUSin+Bla/onUYI3UjhLoKez6dy/+cb8fhUTumfzLgeCXpHEh0obccnxCmN+BQT2YPvkXHtQgghdDV9mIWRKUYaXPCnH5txeKTL/CFFpcGUR7Tb8x6Gylx98wihEyncRdD7YFUhq3fWEG4x8vDZA/SOIzpQVNkKumb/D4D8bhfTHNVd50RCCCFCncmgMONkO3E2hc2VPv613KF3pMA34lrofiJ4HPDFzdJlXoQkKdxFUCtvcPDEt9sAuGtKH9Ji7DonEh3F3FJBr8V/RsFHuS+a8sRxekcSQgghgL1L0WnHJO9scfN1rlvnRAFOUeDsGVqX+V0/w/IX9U4kRIeT5eBEUHvk6y00ODwMSo/m6nGZescRHcXnodfi27A4KmmOyCS/zirj2oUQQgSUE7ua+NNQC/9d5+K+RS0MTDSQFW3UO1ZgqczZ9/5xN8Gip2D+PyG2G8R0O7LtWSMhvof/8gnRgaRwF0FrQXY5szeUYlDg8fMHYTRI4RYqMtY/R/TuFXhN4WQPfxDfgqf1jiSEEELs585RVn7Z7eXnUi83f9/CZ+eGE26W4xXMe3pIzrr+wD/3uuDjq45u27eukeJddEpSuIugVO9w87dZGwG4dnwWA9OjdU4kOkrMrnl02fQSAHljn8ARkaFzIiGEEOLA9o53P+OzJrZV+7htXguvTLFLY0NUOpz3Crhb9v9ZSy389IQ23r3P6dBryuFts64IFj8Dzga/RhWio0jhLoLS499spaTOQde4MO6a0lvvOKKDWBuK6LX0LgBK+1xNVeYZUCWzzwohhAhcyeEGXpli59LZzfy408O/Vzq5f6xN71j6i0o/+M+OuxmWPAc5c6HHyZDYp+NyCaETmZxOBJ1FORV88HMRAE9eOJgwi5yfCgWK10nvxbdgctXTkDCUnSPu0zuSEEIIcVhGpJh46kSte/irG1x8uNWlc6IA1/0kyDwBVJ825t3VrHciIdqdFO4iqDQ43Nz72QYArhmXyXHd43VOJDpK5i//IqJqI25LDDkTZqAaLXpHEkIIIQ7bOT3N3D7CCsADSxwsK/bonCiAKQqMnQ7hidBYBitf0juREO1OCncRVB77ZltrF/m/nirdpkJFwo4vSMl5F4Dc45/FFX6I7nVCCCFEgLpthIWze5rw+OCm75vJrpb1yg/KEgEn3A2KAXYsgB0L9U4kRLuSwl0EjSXbK/ng50IA/n2BdJEPFeFVm+ixQusWv2vQLdSmn6hvICGEEOIoKYrCkxPtDE82Uu+CK+Y0k18nxftBJQ+AwdO02yv+Cw1l+uYRoh1J4S6CQl2zm79+uh6Aq8d2Y2wP6SIfCkwtlfRZeCMGr5Oa9EkUDb5N70hCCCHEMbGZFN44NYy+cQYqmlWumN1McYNP71iBa/AlkNgP3M2w6EnwuvVOJES7kMJddHqqqnL/FxspqXPQLT6Mv57aV+9IogMoPje9F9+KtbmUlshMth//HBiMescSQgghjlmMTeGdM8LoHm2guFHlijnNlDdL8X5ABiNMuFvrOl+ZA6te1zuREO1CCnfR6X22ppg5G0oxGRT+c8kwwq3SRT4UdPvlMaJ3r8RjjiB70qt4LVF6RxJCCCH8JjHMwLtnhpEeoZBf5+PKOc3UOKR4P6CIZDhBWw6W7Dky3l0EJSncRae2s6qJh77cBMAdp/RmaEaMvoFEh0jM/YTU7LcAyB3/DC3RPXVOJIQQQvhfWoSB984MJzFMIbvaxyVfN1PeJMX7AXUZBYMu1m4vnwG1hfrmEcLPpHAXnZbb6+O2D9fR5PIyOiuOmyb20DuS6ABRu1fSfeUDABQN/jM1GafonEgIIYRoP5nRBt4/M4ykPcX7RV81USRj3g9s6OWQOgQ8Tlj4mDbuXYggIYW76LT+b9521hXVEmUz8dy0oRgNit6RRDuz1efTe+HNGHxuKrudzq7Bf9Y7khBCCNHuesUa+fSccDIiFXbWq1z8ZRO5NTLb/H4MRjjhLxAWD3W7YNkMUFW9UwnhF1K4i05paW4lLy7IBeCx8weRHmPXOZFobyZnLX0X/BGzq5aG+CHkjXtaW7tVCCGECAFdowx8ck44PWMMlDapTPuqmU0VUrzvxx4DE+8FxQgFi2Hjx3onEsIvZBYv0emU1rXw5w/W4lPh4pFdOHNwmt6RRDtTvC56//Qn7PX5OMPTyJ70Kj6TTe9YQgghxAHl1vqvK3u4BbKitVVTUsINfHx2GFd/08zGSh8Xf9XEf062c0qm2W/vFxSS+sGYm2DFi7D2HYjOgIgkvVMJcUykcBedisvjY/p7a6hqctE/NYpHzhmodyTR3lSV7isfIHr3CrymcLZNeg23PVHvVEIIIcR+bHuOrG+f3+LX7S64JLy1eI+zaxPW/emHZpYUe7nhuxbuO87H9YMtKIoMG2zV5zRtgrptX8OSZ2DsrXonEuKYSOEuOpXHvtnKmsJaIm0mXr5iBDazrNsd7DLWPUtS3qeoioGcCf9Hc2w/vSMJIYQQB5QaYeDZSTYcHv9sr7jRx4trXTS59n08yqrwv9PC+McyB+9tcfPYCid5tT7+ebwNi1GK91aj/qiNdS9dK+u7i05PCnfRaXy9voQ3lxUA8OzFQ+kaH6ZvINHuUra9RZdNLwKwY8yj1KZP0jmREEIIcWipER0z/4rZqPDo8TZ6xhj453InH21zk1/n44WT7SSFyxwwgDZZ3cR74Ju7oL5Ye8zj1DeTEEdJ/qpFp5Czu4F7PtsAwJ9O7MEp/ZN1TiTaW3z+12SuegSAwiF3Ut7rEp0TCSGEEIFFURSuHWRl5ql2Iszwc6mX0z9rYlmxn5r8g4E1Ak76O5j3TGQ8/5/gk0n9ROcjhbsIeFWNTv7w5iqaXV7Gdo/nzlN66x1JtLPoksX0XHY3Ciqlfa6ieNB0vSMJIYQQAWtSVzNfnh9OnzgDlS0qV8xp5oU1TnyyFJomOh1GXqfdLlgMc+6SZeJEpyNd5UVAc7i93PDOanbVtNAtPowXLx+OySjnm4JZZPkv9Plp71rtZ1Iw6u8gk+0IIYQIYYc7S/2/jrfx8joXPxZ6eHqVkwWFHm4fYSHWtu+x029nqg8Z8T333FBg9f8gMgVOvFfXSEIcCSncRcBSVZV7P9vA6p01RNlMzLx6FHHhFr1jiXYUUb6afvOuxehppjb1eHLHPyVrtQshhAhZxzpL/erdXq785sCv/e1M9SHl+NthyXOw8HFtibiRf9A7kRCHRQp3EbBemJ/LF+tKMBoUXrpiBD2TIvSOJNpRRMUa+s2/FqOnibqUsWSf+Aqq0ap3LCGEEEI3xzJLfVmTj0+y3ZQ2aV3ChyUZOLOHmWqHesCZ6kNG/3PBYIZFT2pd5u2xMOA8vVMJ0SYp3EVA+mp9Cc/8kAPAP88ZyPieCTonEu0pomIt/eZdg8ndSF3yWLZNeh2fya53LCGEEEJ3RztLfVaMgVGpRj7LdvNlroe15T521rs4q4cc/jPpfmjcDWvegk+vA8UI/c/WO5UQhyR/uSLgLMgu586P1gFw3fFZXDamK/mVTTQ5/TNDam55o1+2I/wjonw1/eZfu6doP45tk17D11wN7qPrFrifuiL/bEcIIYToZEwGhWn9LAxLNvLSOhdlTSpvbXYDUN3iA0Kwqzxoc+ec+Zy2NNyGD+HTa+Git6DfmXonE+KgpHAXAWVVQTU3v7saj0/lrCFp3H96P/Irm5j09EK/v5fNLGOn9RZbNI/ei2/B4HVSnzRaa2lvqYFZN/j/zUzS7V4IIURo6h1n5PEJNj7LcfNNngcfcNMPLdx3nMrl/c0YQnESWIMRzv0vqF7Y+Al8cjVc/A70PV3vZEIckBTuImBsLqnjD2+uwuH2MalPIs9ePASjQWltaZ8+qSfpMf7pPm0zG0iNlq7YekrM/YQeK+5HUb3UpE8iZ8ILWvf4+hLtCYMvhvAk/7yZyQrhMtxCCCFE6LKZFC7vbyEzysALa100e+DBJQ4+3ubi7+NsjEoNwbLAYIRzXwbVB5s+g4+vgovfluJdBKQQ/AsVgWhHRSNXzfyZBoeH0Zlx/PfyEZh/t+xbeoydrIRwnRIKv1FV0ja9RLd1TwNQ3uMCdhz3GKrBvO/zwpO0dVeFEEII4Tdpe8bM3zjEwntbXGys9HHRV82c2cPEvWNsdIkMsR6JRhOc96pWvG/+HD66As6eAcMu1zuZEPsIsb9MEYh2VDRy2WsrqWpyMTA9itevGYndEqJjroKc4nWR9fODrUV78YCbyBv75P5FuxBCCCHa1Vk9zCy4JIJL+5pRgNl5Hk7+qJGnfnZQ51T1jtexjCY4/3UYcpnWdf7LP8GS50ENsX8HEdCkcBe6yi5r4OJXVlBW76BnUgRvXTuaKJsUccHI3FJB/x+vICXnfVQU8kc+QOHwv2oTxAghhBCiwyWGGXh8op3ZF4QzJtWI0wsvrnUx4YMGXlrnpMUdQoWr0aSNeR9/m3b/x4fg+wfA59M3lxB7SOEudLNxVx3TXl1OZaOT/qlRfHTDccRHyARiwSiicj2DvjmHqPJf8Jgj2DbpNcr6/UHvWEIIIYQABiQY+fCsMF6ZYqdXrIE6J/x7pZMJHzby9iYXDk+IFPCKAqc8AlMe1e4vfwFmXe+/lW6EOAZSuAtdrN5ZzWWvraC22c2QjBg+uF6K9qCkqiRt/4gB303D2lxGc1QPNp72BbVdTtI7mRBCCCF+Q1EUpmaZmXthOM9MstElUqGiWeXvSx1M/KCR1zc4aQ6VFvhxt8J5r4DBBJs+hf+dBnXFeqcSIU4Kd9Hhftyymytn/kyDU5uI7t3rRhMdJt3jg43JUU3vn26mx4r7MPhcVHc5hU2nzcIR3V3vaEIIIYQ4CKNB4YLeFuZPi+Cfx9tIDVfY3azy6HInx7/fyItrnKExBn7IJXDFLLDHQslaePVEKFyhdyoRwmRWedFhVFVl5pJ8/vXNVlQVTuiVwKtXykR0wSimeAE9lt2DxVGJz2CmaMjtlAy4ERQ5VyiEEEIEgtzatsduD0sy8t9T7Mzf6eGTHDdlTSpPrXIyY42TUzJNnNPTTEq4gXALZEUH4fFc94lww0L48HLYvQnePBNOfwpGXCNz9IgOJ4W76BAer4+HvtrMeysLAbh0dFceOWfAfku+ic7N6Kqn69onScl5H4Dm6F5sP/5ZmuMG6JxMCCGEEAC2PUf/t88/+nHbDi98nefh6zxP62Pzp4XRPSYIS4vYTPjDd9pM81u+hNm3Q8FiOONZsMfoHE6EkiD86xKBpq7Zza0frmVRTgWKAn87vR/XHZ+FImcqg4fqI3HH53Rd8wQWRxUAJX2vpWjYX/CZbDqHE0IIIcReqREGnp1kw+Fp+7kHoqoqubU+lhR72V7za6v9td+28MfBVs7rZSbCEmTHeNYIuOgtWPIczH8UNn0GRT9r4+Azx+udToQIKdxFu1pXVMst769hV00LdrOR5y8ZytQBKXrHEn4UXrWJrFX/ILJiDQAtUd3ZMfoR6lPH6ZxMCCGEEAeSGnFsPR67xxqZkmWmqN7Hx9lufinzsrNe5cElDv690sE5vcxc0NvMsCRj8DTUKAqccCdkTYDP/gg1+fDWmXD8nTDxHjBZ9E4ogpz0UxbtQlVVXl+8g4teXsaumha6xoXxyU1jpWgPIra6PHouuYNB35xDZMUavKYwdg6/l/VnfiNFuxBCCBECMqIMnN9Lm2D4+sEWukcbaHTDe1vcnP9FMyd/3MSLa5yUNAbRWuhdRsJNi2HoFaD6YPHT8MoJsHOZ3slEkJMWd+F3tc0u7v5kAz9u3Q3A6YNSeOKCwUTZZOb4YGCr20GXjTNIKPgaRdW+iCszz2TniPtxhcmJGSGEECIUndPTzP3HWVle4uWTbDdz893sqPXx1ConT69yMi7dyAW9zZyaZSbM3Mlb4a2RcO6L0GsyzLkbKrZpS8YNvUJbBz48Xu+EIghJ4S786tuNpTz45WYqG51YjAYeOLMfVx7XLXi6SYUqVSWqbBkpOe8SV/RDa8Fe3eUUiob8WSafE0IIIQSKojAu3cS4dBP/dNn4ZoebWdvdrCjxsrRYuzyw2MFp3c2c2UN7ntXYiY8RB5wHWRNh3sOw+k1Y9y5kfwMn/Q2GXw1GabQS/iOFu/CL8noHf/9yM3M3lwHQIzGc56cNY1CXaJ2TiWNhdNWTmDeLlJx3sdfvaH1cCnYhhBBCHEqEReHivhYu7muhqMHH5zluPstxsbNe5bMcN5/luIm0wEldTZyaZWZihqlztsSHxcFZ/4Ehl8HsO6B8M8y5C5a9ACc9AAPOB4OMThbHTgp3cUy8PpWPVhXxxLdbqXd4MBkU/nRiD6af1BOrKQjX8wwBBncjcUU/Er9zDjElizH4XAB4zBFUdj+Pst6X0xLTW+eUQgghhOgsMiIN/HmElVuHW1i928uX2918V+ChvFnly1wPX+Z6sJlgYhcTp2aZOKmbmWhrJyviu46BG3+CX/4Hi57UJq/77DpY+jxMegB6TZECXhwTKdzFUfspp4LH5mwle3cDAIPSo3nywsH0S43SOZk4UtbGXUSXLCK2ZBExJT9h8Dpbf9YU04fdvS+novu5+MwR+7+4vhjcR78W7H7qivy3LSGEEEJ0iNzaw5uAzmZUmNbXwkV9zGRX+1he4mVpsYfdzSrfFXj4rsCDSXEwMNHAiGQTw5ONdI1Ujn3YZZ2JcF8KWZU5x7adtmSMhovfho2fwvoPoGwjfDANYrrB4GnQczKYrAd/vTUS4nu0b0bRKUnhLo7Y1tJ6HvtmK4u3VwIQbTdz28m9uGpsN0xGOZPYGViaSoisWEtk+SpiShdjr8/f5+ctUVlUdTuTyswzDt26Xl8Ms25on5CH+lITQgghRECw7akmbp/vv5P4HhXWlftYV+5i5kZ/bTUOeJYFn95JlqHMXxs9fLU7tZb4RU+2/dxb10jxLvYjhbs4bGsKa/jvgrzW2eLNRoWrx2Zyy0k9iQmTtSsDkqpiaS4hrCabsNpswqu3EFmxBmtz6b5PU4w0JA6jNnUCNV1Oojm2n7ZeaVv2trQPvhjCk/yX22SF8AT/bU8IIYQQ7SI1wsCzk2w4PP7bZkWzj5wa7ZJf58Pzm8Z8A5ARpdAr1kj3aANdIhVMhraPWYobfby41kXTCQ9ArB/DHg63A4qWw46fwFG750EFEvtoLfTJg7SJ7OqKYPEz4Gzo2HyiU5DCXRySz6eyOLeS/y7IZWV+NaDVc6cPSuWvU/vQLT58v9fkVzbR5PTfB2JueaPfttWu/N1lHMBsh6j0w3qq0VlHWF3OniJ9257rHEzu/T/8VcVAU2QPGmL7Ux8/hLr4YXjNe/4vVaA67/Dy7e3WHp4E0YeXUwghhBDBJTXCvz0us2IMjE7Tbru8KlurfKwv97K+wktJo8rOepWd9dqxptkAvWIN9Is30j/eQM9YA5ZDzVQfnQHxOszDlDIQRlwLBUsgZy7s3qQtI1exDSwRkHEcxGZ2fC7RaUjhLg6ovN7BJ6t38fEvReysaga0FvbzhqVzw4Qe9Ew6wFhntKJ90tML2yWTzRzA3fDbs8v4+a+2Fu9GVz22hgJs9QXY91zbGgqwNezE7Kw54Mt9iglHVHeaIzJoKtpAo2qnETu+GgPU5MCOHOCTY8so3dqFEEII0Q4sRoUhSUaGJGnFdkWzjw0VPjZWeNla5aXeBVuqfGyp8vEZYDJAzxgD/eIN9Ikz0iPGQIQlQCa6M5ig+4napb4U8n6E3HnQXKnd3uvHh2DQxdB9IkR30SutCDBSuItWzS4PC7Mr+GJtMfO2leP1qQBEWE1cPDKDP56QRVqM/ZDb2NvSPn1ST9LbeO6RsJkNpEb7b3t+58cu4wavA7ujHFtdLrbCRdh+eRi7qxpbfQFmZ/UhX+sMS6U5tg/NMXsusX1oieqBarRAVS4U3ibd2oUQQgjRaSWGGTi5m4GTu5lQVZWSRpUtVV62VvnYWuWj1qmyrdrHtmofoB2XpoYrpIRrxfv2Gi+949pole8IUakw7EptGbnyzVC4HPIXa13pdyzULgDxPbVCP/N46DJKCvkQJoV7iKtrcbMwu5xvN5axMKcch/vXQUQjusUybVQGZw5OJcxyZLtKeoydrIT9u9EHvcPsMm7wOrE1l2JrLsHWXIK99XYpFlftb54IFP+4z2td1jgcYWk4wtNp2XPtCE/HEZaGz/S7kxs+oLZQuy3d2oUQQggRRBRFIT1SIT3SwCmZoKoqZU3qniLeS26tj7ImldI9F4A7Fji4Z5GDgQlGhiYZGZBgoH+81jKvSzFvMELKYO3S/SSYcwcMuwoqtkLxaq3hpSoXVr2uPT8yDbqMhPQRWvf75EEQmXzo96jK8++4eZn5XhdSuIcYh9vLmsIaluZWsiS3io27atnTsA5A17gwThuYwgUjutA7OVK/oEFA8bqwtZTtV5jbmkuxOqsO+Vq3OZoWWwKOukocWHBgoUW14MCMr9kIzV6oLAQKjzyYdGsXQgghRBBSFIXUCIXUCAMnddPKnAaXSl6Nj1VlHuYXeokwQ6Mb1uz2sma3t/W1vx8r3z/BSP94Y8euJ793YuBR10HaUGiphZ1Ltdb3opVQtgkaSmDrV9plr/BESOqvFdNx3SE2a891JjSUwozh/s8qM993OCncg5jXp7Kzqon1u2pZV1jLuqJatpTW4/aq+zyvR2I4pw1M5dSBKQxIizr2dTJDiMHjwNpYiG33CmxKFbb8D7B567E3l2JxVKKgHvS1HlMELWGpWut5WNpvbqfg3bteelMleJwH3cYRk27tQgghhAghkRaFoclaAT6/0MsHZ4YRblH2THbn29PN3kvD78bK75USrtAjxkD3aAM9YrWW+R4xBlLD/bC2fFvsMdD3DO0C4GqCknWw62ftevdmrTW+qQLyf9Iuvxe257gveQBEdQFbFNiiwRq15xKptfofLpn5XjdSuAeBRqeHwqpmCqubKahqIqesgZzyBrbvbsT52/Uz9kiMtHJ8zwTG90xgfM/4wB47HgAM7mZsjTuxNezcMxncztbLPsuqGYCK8n1e6zGF4bDvLchTf1Ocp+KxRLX95lJkCyGEEEL4jaIodI8x0j3GyHm9tcdUVWVXo8qWSm2s/JYqL1uqvOxq0LrelzV5WVrsBdyt2wkzQfcYA91jDHSNNNDlN5fUCAVre3S7t4RD5njtsperWetWX74VqndAdb52XZMPjjpt4jvQivzdmw/0L6IV8mFxYI/9tZjf5/Kbx8Li/f97icMSNIX7iy++yFNPPUVZWRlDhgxhxowZjB49Wu9YflPX4ubHLbvZUlpPWZ2DqkYnlU0uKhqc1LW4D/o6i9FA98Rw+qRE0js5kj7JkSRHWVvPEFY1unC4faE5Hh1QfG7MzeVYm0uwNJVibS7F2qTdtuy5fbDZ2vfymCNxhKXiqCnD0WUcjtjerUW6xxx9eOuhCyGEEEKIdpdbu3+j1l7pEQbSIwxM3tPNvtGlsqvRx64GH7saVO260Udpo0qzBzZV+thUuf/2FCA5XNEK+QgDKREKiXaFxDADSWEKiWEKiXYDkRaOvdXeEqaNd08fsf/Pmqsh5zv44ibofRqgao+1VGvXjlpQfdp16/ryh+mNqVrBb4nYt8i3RIB1z2OWSO222Q4mO5htYNpzMdt/c9sGDbvB69J6hxr8UKIG4Tj8oCjcP/roI+68805efvllxowZw/PPP8/UqVPJzs4mKcmPs2frqKbJxV2frD/i17m8PraVNbCt7NDdWRbcfWLnLt59XoyeJozuRoyuBozuRkzuRoyueszOasyOakyOyj23qzA7tGuTq+6wNu+2xuKI7PabS6Z2HZWJxxKjrXv+9W3Q5SyZ+E0IIYQQIsDY9lQ9t89v8et2bxxiptkNuxr3FPYNPlo8tLbU/4L3oK+1mSDRrpBgNxBrU4gmimj3VUSvqCc6KZ9ou5mYMDPRdjMRNhPhFhN2i5Fwiwmb2dB20R8WB0n9tNu9p2oz1P+WzwvO+l+L+ZYarQv8Ppd6cDWCo16779vTYOhxQKMD2H30/3jt7bxXYMgleqfwm6Ao3J999lmuv/56rr32WgBefvll5syZwxtvvMG9996rczr/SLH7GJngoaSyhoFJFlLDIc7sJc7qJcnqJcykne070JhqRf3tY+o+1zVNLn7YUoZpcwXEhoGq/u55HNFjMbUtXGzMoXvRdhJrLPu8r6J6UXxeUL17bnu069bHPSg+L4rqQVF9KD43Bq+z9aJ4nRi8jt885sDobtaKdU/TEf6L/spnsOAKS8EZnoorLBVneBqu8FScYanadXg63sPp1i6EEEIIIQJSaoSBZyfZcHj8s73iRh8vrnVxVg8LAxN/HSOuqirVDrW1hb6owcfuJpWKFh/lzSqVzdrtBhc4PFDUoFLUsLe4twGnws8NwJZDvr+iQJjZiN1iIsxiJMxixGIyYDYaMBsVLCYjFqOC2VWP2TUd86pILGEtWAxgNiqYDdqEfIpix6h0waB0QVHAoIDRBAYzKBF77itK688MDWUYNn2MYcwNGMLjUDxOFI8DPC1aMe92oHid2lLJHge4W1B8bq013esBnxM8bvC5wOtG8bq0n/m0/5jf1zL736eNn/96v3/BDroNafO/stNQVFU9+OxZnYDL5SIsLIxPP/2Uc889t/Xxq6++mtraWr788sv9XuN0OnE6f53wq66ujq5du1JUVERUVIAWaLW74KXj9E4R8LwYcRts2sVow2Ow4zSG4zRF4DBG4jRG4DRF7LmOxGmIwGm0g2I4tjduqoCCRZA5Ucb+CCGEEEIEuZ1NRt7KC+eqHs10jzzyswFuLzR4FRrdBprcCi1eBUdLM81Vu3BEZtJiCMPhVWjxKDh8Ck4vuHwKbp8MwTxc9w9p4bLzz9U7xiHV19eTkZFBbW0t0dHRh3xup29xr6ysxOv1kpy87/qFycnJbNu27YCvefzxx3n44Yf3ezwjI6NdMopQ8o3eAYQQQgghRAf5l94BxEHdDNx8rd4pDk9DQ0PwF+5H47777uPOO+9sve/z+aiuriY+Pl6WQtPJ3rNNAd3rQYQM2R9FoJF9UgQS2R9FoJF9UgSSI9kfVVWloaGBtLS0Nrfb6Qv3hIQEjEYju3fvOzHC7t27SUlJOeBrrFYrVqt1n8diYmLaK6I4AlFRUfKBKwKG7I8i0Mg+KQKJ7I8i0Mg+KQLJ4e6PbbW073WMA3v1Z7FYGDFiBPPmzWt9zOfzMW/ePMaOHatjMiGEEEIIIYQQ4th1+hZ3gDvvvJOrr76akSNHMnr0aJ5//nmamppaZ5kXQgghhBBCCCE6q6Ao3KdNm0ZFRQV///vfKSsrY+jQocydO3e/CetE4LJarTz00EP7DWEQQg+yP4pAI/ukCCSyP4pAI/ukCCTttT92+uXghBBCCCGEEEKIYNbpx7gLIYQQQgghhBDBTAp3IYQQQgghhBAigEnhLoQQQgghhBBCBDAp3IUQQgghhBBCiAAmhbvoMC+++CKZmZnYbDbGjBnDzz//fNDnbt68mQsuuIDMzEwUReH555/vuKAiJBzJ/vjaa69xwgknEBsbS2xsLJMnTz7k84U4GkeyT86aNYuRI0cSExNDeHg4Q4cO5Z133unAtCLYHcn++FsffvghiqJw7rnntm9AEXKOZJ988803URRln4vNZuvAtCLYHelnZG1tLdOnTyc1NRWr1Urv3r355ptvjug9pXAXHeKjjz7izjvv5KGHHmLNmjUMGTKEqVOnUl5efsDnNzc30717d5544glSUlI6OK0Idke6Py5cuJBLL72UBQsWsHz5cjIyMpgyZQrFxcUdnFwEqyPdJ+Pi4vjb3/7G8uXL2bBhA9deey3XXnst3333XQcnF8HoSPfHvQoKCrj77rs54YQTOiipCBVHs09GRUVRWlraetm5c2cHJhbB7Ej3R5fLxSmnnEJBQQGffvop2dnZvPbaa6Snpx/ZG6tCdIDRo0er06dPb73v9XrVtLQ09fHHH2/ztd26dVOfe+65dkwnQs2x7I+qqqoej0eNjIxU33rrrfaKKELMse6Tqqqqw4YNUx944IH2iCdCzNHsjx6PRx03bpz6+uuvq1dffbV6zjnndEBSESqOdJ/83//+p0ZHR3dQOhFqjnR/fOmll9Tu3burLpfrmN5XWtxFu3O5XKxevZrJkye3PmYwGJg8eTLLly/XMZkIRf7YH5ubm3G73cTFxbVXTBFCjnWfVFWVefPmkZ2dzYQJE9ozqggBR7s/PvLIIyQlJXHdddd1REwRQo52n2xsbKRbt25kZGRwzjnnsHnz5o6IK4Lc0eyPX331FWPHjmX69OkkJyczcOBAHnvsMbxe7xG9txTuot1VVlbi9XpJTk7e5/Hk5GTKysp0SiVClT/2x3vuuYe0tLR9PrSFOFpHu0/W1dURERGBxWLhjDPOYMaMGZxyyintHVcEuaPZH5csWcLMmTN57bXXOiKiCDFHs0/26dOHN954gy+//JJ3330Xn8/HuHHj2LVrV0dEFkHsaPbHHTt28Omnn+L1evnmm2948MEHeeaZZ3j00UeP6L1NR51aCCFC0BNPPMGHH37IwoULZaIboavIyEjWrVtHY2Mj8+bN484776R79+6ceOKJekcTIaShoYErr7yS1157jYSEBL3jCAHA2LFjGTt2bOv9cePG0a9fP1555RX++c9/6phMhCKfz0dSUhKvvvoqRqORESNGUFxczFNPPcVDDz102NuRwl20u4SEBIxGI7t3797n8d27d8vEc6LDHcv++PTTT/PEE0/w448/Mnjw4PaMKULI0e6TBoOBnj17AjB06FC2bt3K448/LoW7OCZHuj/m5eVRUFDAWWed1fqYz+cDwGQykZ2dTY8ePdo3tAhq/jiONJvNDBs2jNzc3PaIKELI0eyPqampmM1mjEZj62P9+vWjrKwMl8uFxWI5rPeWrvKi3VksFkaMGMG8efNaH/P5fMybN2+fs6FCdISj3R+ffPJJ/vnPfzJ37lxGjhzZEVFFiPDXZ6TP58PpdLZHRBFCjnR/7Nu3Lxs3bmTdunWtl7PPPptJkyaxbt06MjIyOjK+CEL++Iz0er1s3LiR1NTU9oopQsTR7I/jx48nNze39aQmQE5ODqmpqYddtAMyq7zoGB9++KFqtVrVN998U92yZYt6ww03qDExMWpZWZmqqqp65ZVXqvfee2/r851Op7p27Vp17dq1ampqqnr33Xera9euVbdv367XryCCyJHuj0888YRqsVjUTz/9VC0tLW29NDQ06PUriCBzpPvkY489pn7//fdqXl6eumXLFvXpp59WTSaT+tprr+n1K4ggcqT74+/JrPLC3450n3z44YfV7777Ts3Ly1NXr16tXnLJJarNZlM3b96s168ggsiR7o+FhYVqZGSkesstt6jZ2dnq7Nmz1aSkJPXRRx89oveVrvKiQ0ybNo2Kigr+/ve/U1ZWxtChQ5k7d27rxA6FhYUYDL92ACkpKWHYsGGt959++mmefvppJk6cyMKFCzs6vggyR7o/vvTSS7hcLi688MJ9tvPQQw/xj3/8oyOjiyB1pPtkU1MTf/rTn9i1axd2u52+ffvy7rvvMm3aNL1+BRFEjnR/FKK9Hek+WVNTw/XXX09ZWRmxsbGMGDGCZcuW0b9/f71+BRFEjnR/zMjI4LvvvuOOO+5g8ODBpKenc9ttt3HPPfcc0fsqqqqqfv1NhBBCCCGEEEII4TdyulQIIYQQQgghhAhgUrgLIYQQQgghhBABTAp3IYQQQgghhBAigEnhLoQQQgghhBBCBDAp3IUQQgghhBBCiAAmhbsQQgghhBBCCBHApHAXQgghhBBCCCECmBTuQgghhBBCCCFEAJPCXQghhBAB78QTT+T222/XO4YQQgihCynchRBCCB0sX74co9HIGWecoXeUDjN79mwmTpxIZGQkYWFhjBo1ijfffHOf5yxcuBBFUaitrdUloxBCCBGIpHAXQgghdDBz5kxuvfVWFi1aRElJSbu+l6qqeDyedn2PtsyYMYNzzjmH8ePHs3LlSjZs2MAll1zCTTfdxN13361LJpfLpcv7CiGEEEdKCnchhBCigzU2NvLRRx9x8803c8YZZ+zT6nzZZZcxbdq0fZ7vdrtJSEjg7bffBsDn8/H444+TlZWF3W5nyJAhfPrpp63P39tq/e233zJixAisVitLliwhLy+Pc845h+TkZCIiIhg1ahQ//vjjPu9VWlrKGWecgd1uJysri/fff5/MzEyef/751ufU1tbyxz/+kcTERKKiojjppJNYv379QX/foqIi7rrrLm6//XYee+wx+vfvT8+ePbnrrrt46qmneOaZZ1i5ciUFBQVMmjQJgNjYWBRF4Zprrmndjs/n469//StxcXGkpKTwj3/8Y5/3aSvXP/7xD4YOHcrrr79OVlYWNpsNgE8//ZRBgwZht9uJj49n8uTJNDU1Hfw/UAghhOhgUrgLIYQQHezjjz+mb9++9OnThyuuuII33ngDVVUBuPzyy/n6669pbGxsff53331Hc3Mz5513HgCPP/44b7/9Ni+//DKbN2/mjjvu4IorruCnn37a533uvfdennjiCbZu3crgwYNpbGzk9NNPZ968eaxdu5ZTTz2Vs846i8LCwtbXXHXVVZSUlLBw4UI+++wzXn31VcrLy/fZ7kUXXUR5eTnffvstq1evZvjw4Zx88slUV1cf8Pf99NNPcbvdB2xZv/HGG4mIiOCDDz4gIyODzz77DIDs7GxKS0v5z3/+0/rct956i/DwcFauXMmTTz7JI488wg8//HBEuXJzc/nss8+YNWsW69ato7S0lEsvvZQ//OEPbN26lYULF3L++ee3/n8IIYQQAUEVQgghRIcaN26c+vzzz6uqqqput1tNSEhQFyxYsM/9t99+u/X5l156qTpt2jRVVVXV4XCoYWFh6rJly/bZ5nXXXadeeumlqqqq6oIFC1RA/eKLL9rMMmDAAHXGjBmqqqrq1q1bVUBdtWpV68+3b9+uAupzzz2nqqqqLl68WI2KilIdDsc+2+nRo4f6yiuvHPA9brrpJjU6OvqgGQYPHqyedtpp+2SvqanZ5zkTJ05Ujz/++H0eGzVqlHrPPfccdq6HHnpINZvNanl5eevPV69erQJqQUHBQfMJIYQQejPpetZACCGECDHZ2dn8/PPPfP755wCYTCamTZvGzJkzOfHEEzGZTFx88cW89957XHnllTQ1NfHll1/y4YcfAlqLcXNzM6eccso+23W5XAwbNmyfx0aOHLnP/cbGRv7xj38wZ84cSktL8Xg8tLS0tLa4Z2dnYzKZGD58eOtrevbsSWxsbOv99evX09jYSHx8/D7bbmlpIS8v7xj/dQ5t8ODB+9xPTU1t7Q1wuLm6detGYmJi6/0hQ4Zw8sknM2jQIKZOncqUKVO48MIL9/mdhRBCCL1J4S6EEEJ0oJkzZ+LxeEhLS2t9TFVVrFYrL7zwAtHR0Vx++eVMnDiR8vJyfvjhB+x2O6eeeipAaxf6OXPmkJ6evs+2rVbrPvfDw8P3uX/33Xfzww8/8PTTT9OzZ0/sdjsXXnjhEU3S1tjYSGpqKgsXLtzvZzExMQd8Te/evamrq6OkpGSf3xu0Ew55eXmtY9sPxWw273NfURR8Pt8R5fr9v4nRaOSHH35g2bJlfP/998yYMYO//e1vrFy5kqysrDYzCSGEEB1BxrgLIYQQHcTj8fD222/zzDPPsG7dutbL+vXrSUtL44MPPgBg3LhxZGRk8NFHH/Hee+9x0UUXtRat/fv3x2q1UlhYSM+ePfe5ZGRkHPL9ly5dyjXXXMN5553HoEGDSElJoaCgoPXnffr0wePxsHbt2tbHcnNzqampab0/fPhwysrKMJlM+71/QkLCAd/3ggsuwGw288wzz+z3s5dffpmmpiYuvfRSACwWCwBer/cw/kV/dTS59lIUhfHjx/Pwww+zdu1aLBZLa48IIYQQIhBIi7sQQgjRQWbPnk1NTQ3XXXcd0dHR+/zsggsuYObMmdx0002ANrv8yy+/TE5ODgsWLGh9XmRkJHfffTd33HEHPp+P448/nrq6OpYuXUpUVBRXX331Qd+/V69ezJo1i7POOgtFUXjwwQdbW6wB+vbty+TJk7nhhht46aWXMJvN3HXXXdjtdhRFAWDy5MmMHTuWc889lyeffJLevXtTUlLCnDlzOO+88/brng/QtWtXnnzySe666y5sNhtXXnklZrOZL7/8kvvvv5+77rqLMWPGAFpXdkVRmD17Nqeffjp2u52IiIg2/22PJhfAypUrmTdvHlOmTCEpKYmVK1dSUVFBv3792nxPIYQQoqNIi7sQQgjRQWbOnMnkyZP3K9pBK9x/+eUXNmzYAGizy2/ZsoX09HTGjx+/z3P/+c9/8uCDD/L444/Tr18/Tj31VObMmdNm1+5nn32W2NhYxo0bx1lnncXUqVP3Gc8O8Pbbb5OcnMyECRM477zzuP7664mMjGxdOk1RFL755hsmTJjAtddeS+/evbnkkkvYuXMnycnJB33v22+/nc8//5zFixczcuRIBg4cyPvvv89LL73E008/3fq89PR0Hn74Ye69916Sk5O55ZZbDv2PusfR5oqKimLRokWcfvrp9O7dmwceeIBnnnmG00477bDeVwghhOgIiqrKeidCCCGEOLBdu3aRkZHBjz/+yMknn6x3HCGEECIkSeEuhBBCiFbz58+nsbGRQYMGUVpayl//+leKi4vJycnZb3I4IYQQQnQMGeMuhBBCiFZut5v777+fHTt2EBkZybhx43jvvfekaBdCCCF0JC3uQgghhBBCCCFEAJPJ6YQQQgghhBBCiAAmhbsQQgghhBBCCBHApHAXQgghhBBCCCECmBTuQgghhBBCCCFEAJPCXQghhBBCCCGECGBSuAshhBBCCCGEEAFMCnchhBBCCCGEECKASeEuhBBCCCGEEEIEsP8Hfzi6yIOSPd0AAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA+0AAAK9CAYAAABRvo1QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAACN50lEQVR4nOzdd3RUdf7/8dednh5CCAkQIBQRC4KKig2CKGJZXXUta1/L/hRdUdevq669rWvXxdUtirq6rrq76NrpKnYUEESQ3gmQXibT7u+Pm5lkSAJJmGQmyfNxzpy5c++dez+TieWV96cYpmmaAgAAAAAACccW7wYAAAAAAICmEdoBAAAAAEhQhHYAAAAAABIUoR0AAAAAgARFaAcAAAAAIEER2gEAAAAASFCEdgAAAAAAEhShHQAAAACABEVoBwAAAAAgQRHaAQBd2l133SXDMDrkXuPGjdO4ceMir+fOnSvDMPTmm292yP0vueQSDRw4sEPu1VaVlZW6/PLLlZubK8MwNGXKlHg3qd3E+vvY9fcLANA9ENoBAJ3GtGnTZBhG5OHxeNSnTx9NnDhRTz31lCoqKmJyn82bN+uuu+7SwoULY3K9WErktrXEAw88oGnTpumqq67Syy+/rAsvvLDROeE/tOzpsbsAu3btWl166aUaPHiwPB6PcnNzdeyxx+rOO++MOu+ZZ57RtGnT2vx5Yv19/PDDD7rrrru0du3amFwPAND5GaZpmvFuBAAALTFt2jRdeumluueee1RQUCC/36+tW7dq7ty5mjFjhvr376+3335bI0aMiLwnEAgoEAjI4/G0+D7ffPONRo8erRdeeEGXXHJJi9/n8/kkSS6XS5JVaS8sLNQbb7yhs846q8XXaWvb/H6/QqGQ3G53TO7VHo444gg5HA59+umnzZ6zePFiLV68OPK6srJSV111lX7+85/rjDPOiOzv3bu3jj/++EbvX7lypUaPHq2kpCT96le/0sCBA7VlyxZ9++23ev/99+X1eiPnHnDAAcrOztbcuXPb9Hli/X28+eab+sUvfqE5c+Y0+qPErr9fAIDuwRHvBgAA0FqTJk3SoYceGnl9yy23aPbs2TrllFP0s5/9TMuWLVNSUpIkyeFwyOFo3//cVVdXKzk5Oe5hyul0xvX+LVFUVKT99ttvt+eMGDEi6g8vO3bs0FVXXaURI0boggsu2OM9Hn/8cVVWVmrhwoUaMGBAo/t3lFh/H/H+/QIAxAfd4wEAXcL48eN1++23a926dfrHP/4R2d/UmPYZM2bo6KOPVmZmplJTUzVs2DDdeuutkqzq+OjRoyVJl156aaQrdrgL9bhx43TAAQdowYIFOvbYY5WcnBx5b3NjjoPBoG699Vbl5uYqJSVFP/vZz7Rhw4aocwYOHNhkVb/hNffUtqbGUFdVVenGG29Ufn6+3G63hg0bpkceeUS7drQzDEPXXHONpk+frgMOOEBut1v777+/Pvjgg6Z/4LsoKirSZZddpt69e8vj8eiggw7Siy++GDkeHt+/Zs0avfvuu5G2t0c38FWrVqlfv36NArsk5eTkRLYHDhyopUuXat68eY263BcXF+u3v/2tDjzwQKWmpio9PV2TJk3SokWLoj5Ta7+P1157TYcccojS0tKUnp6uAw88UE8++aQkqyfJL37xC0lSYWFh5HrhXgBN/X55vV7ddddd2meffeTxeJSXl6czzjhDq1atauuPDwCQYKi0AwC6jAsvvFC33nqrPvroI11xxRVNnrN06VKdcsopGjFihO655x653W6tXLlS8+fPlyQNHz5c99xzj+644w5deeWVOuaYYyRJRx55ZOQaO3fu1KRJk3TuuefqggsuUO/evXfbrvvvv1+GYejmm29WUVGRnnjiCU2YMEELFy6M9AhoiZa0rSHTNPWzn/1Mc+bM0WWXXaaRI0fqww8/1E033aRNmzbp8ccfjzr/008/1X/+8x9dffXVSktL01NPPaUzzzxT69evV8+ePZttV01NjcaNG6eVK1fqmmuuUUFBgd544w1dcsklKi0t1XXXXafhw4fr5Zdf1vXXX69+/frpxhtvlCT16tWrxZ+/pQYMGKCZM2dq9uzZGj9+fLPnPfHEE7r22muVmpqq2267TZIi3+Xq1as1ffp0/eIXv1BBQYG2bdum5557TmPHjtUPP/ygPn36tPr7mDFjhs477zwdd9xxeuihhyRJy5Yt0/z583Xdddfp2GOP1W9+8xs99dRTuvXWWzV8+HBJijzvKhgM6pRTTtGsWbN07rnn6rrrrlNFRYVmzJihJUuWaPDgwW37AQIAEosJAEAn8cILL5iSzK+//rrZczIyMsxRo0ZFXt95551mw//cPf7446Ykc/v27c1e4+uvvzYlmS+88EKjY2PHjjUlmc8++2yTx8aOHRt5PWfOHFOS2bdvX7O8vDyy//XXXzclmU8++WRk34ABA8yLL754j9fcXdsuvvhic8CAAZHX06dPNyWZ9913X9R5Z511lmkYhrly5crIPkmmy+WK2rdo0SJTkvn00083uldDTzzxhCnJ/Mc//hHZ5/P5zDFjxpipqalRn33AgAHmySefvNvr7Wr79u2mJPPOO+9s0flLliwxk5KSTEnmyJEjzeuuu86cPn26WVVV1ejc/fffP+rnG+b1es1gMBi1b82aNabb7TbvueeeyL7WfB/XXXedmZ6ebgYCgWbb/sYbb5iSzDlz5jQ6tuvvwvPPP29KMh977LFG54ZCoWbvAQDoXOgeDwDoUlJTU3c7i3xmZqYk6a233lIoFGrTPdxuty699NIWn3/RRRcpLS0t8vqss85SXl6e3nvvvTbdv6Xee+892e12/eY3v4naf+ONN8o0Tb3//vtR+ydMmBBVnR0xYoTS09O1evXqPd4nNzdX5513XmSf0+nUb37zG1VWVmrevHkx+DQtt//++2vhwoW64IILtHbtWj355JM6/fTT1bt3b/31r39t0TXcbrdsNut/k4LBoHbu3BkZSvHtt9+2qV2ZmZmqqqrSjBkz2vT+Xf373/9Wdna2rr322kbHOmqZQwBA+yO0AwC6lMrKyqiAvKtzzjlHRx11lC6//HL17t1b5557rl5//fVWBfi+ffu2alKwoUOHRr02DENDhgxp92W91q1bpz59+jT6eYS7W69bty5qf//+/Rtdo0ePHiopKdnjfYYOHRoJuXu6T6xs3bo16lFTUxM5ts8+++jll1/Wjh07tHjxYj3wwANyOBy68sorNXPmzD1eOxQK6fHHH9fQoUPldruVnZ2tXr16afHixSorK2tTe6+++mrts88+mjRpkvr166df/epXLZ4zoCmrVq3SsGHD2n2iRQBAfBHaAQBdxsaNG1VWVqYhQ4Y0e05SUpI+/vhjzZw5UxdeeKEWL16sc845R8cff7yCwWCL7tOacegt1VxltKVtigW73d7kfjNBV4fNy8uLevzrX/9qdI7dbteBBx6oW265Rf/9738lSa+88soer/3AAw/ohhtu0LHHHqt//OMf+vDDDzVjxgztv//+be6hkZOTo4ULF+rtt9+OzDUwadIkXXzxxW26HgCge+BPswCALuPll1+WJE2cOHG359lsNh133HE67rjj9Nhjj+mBBx7Qbbfdpjlz5mjChAkx71r8008/Rb02TVMrV66MWtasR48eKi0tbfTedevWadCgQZHXrWlbeEK2ioqKqGr7jz/+GDkeCwMGDNDixYsVCoWiqu2xvs+udu1mvv/+++/2/PAygVu2bInsa+7n+eabb6qwsFB///vfo/aXlpYqOzt7j+9vjsvl0qmnnqpTTz1VoVBIV199tZ577jndfvvtGjJkSKuuN3jwYH355Zfy+/2dYrk/AEDbUGkHAHQJs2fP1r333quCggKdf/75zZ5XXFzcaN/IkSMlSbW1tZKklJQUSWoyRLfFSy+9FDXO/s0339SWLVs0adKkyL7Bgwfriy++kM/ni+x75513Gi0N15q2nXTSSQoGg/rTn/4Utf/xxx+XYRhR998bJ510krZu3RpV6Q4EAnr66aeVmpqqsWPHxuQ+u5owYULUIy8vT5L0ySefyO/3Nzo/PIfAsGHDIvtSUlKa/Fna7fZGPQzeeOMNbdq0KWpfa76PnTt3Rr222WyRP9y05XfvzDPP1I4dOxp9v1Li9o4AALQelXYAQKfz/vvv68cff1QgENC2bds0e/ZszZgxQwMGDNDbb78tj8fT7Hvvueceffzxxzr55JM1YMAAFRUV6ZlnnlG/fv109NFHS7ICdGZmpp599lmlpaUpJSVFhx9+uAoKCtrU3qysLB199NG69NJLtW3bNj3xxBMaMmRI1LJ0l19+ud58802deOKJOvvss7Vq1Sr94x//aLRsV2vaduqpp6qwsFC33Xab1q5dq4MOOkgfffSR3nrrLU2ZMiVmS4JdeeWVeu6553TJJZdowYIFGjhwoN58803Nnz9fTzzxxG7nGGgPDz30kBYsWKAzzjgjEoq//fZbvfTSS8rKytKUKVMi5x5yyCH685//rPvuu09DhgxRTk6Oxo8fr1NOOUX33HOPLr30Uh155JH6/vvv9corr0T1epBa931cfvnlKi4u1vjx49WvXz+tW7dOTz/9tEaOHBkZ/z9y5EjZ7XY99NBDKisrk9vt1vjx46PWlw+76KKL9NJLL+mGG27QV199pWOOOUZVVVWaOXOmrr76ap122mkx/KkCAOImrnPXAwDQCuEl38IPl8tl5ubmmscff7z55JNPRi0tFrbrkm+zZs0yTzvtNLNPnz6my+Uy+/TpY5533nnmihUrot731ltvmfvtt5/pcDiilvQaO3asuf/++zfZvuaWfPvnP/9p3nLLLWZOTo6ZlJRknnzyyea6desavf/RRx81+/bta7rdbvOoo44yv/nmm0bX3F3bdl1izDRNs6Kiwrz++uvNPn36mE6n0xw6dKj58MMPN1oSTJI5efLkRm1qbim6XW3bts289NJLzezsbNPlcpkHHnhgk8ugdcSSb/PnzzcnT55sHnDAAWZGRobpdDrN/v37m5dccom5atWqqHO3bt1qnnzyyWZaWpopKfKz9nq95o033mjm5eWZSUlJ5lFHHWV+/vnne/V9vPnmm+YJJ5xg5uTkmC6Xy+zfv7/561//2tyyZUvU9f7617+agwYNMu12e9Tyb03du7q62rztttvMgoIC0+l0mrm5ueZZZ53V6HMCADovwzTpPwUAAAAAQCJiTDsAAAAAAAmK0A4AAAAAQIIitAMAAAAAkKAI7QAAAAAAJChCOwAAAAAACYrQDgAAAABAgnLEuwGJIBQKafPmzUpLS5NhGPFuDgAAAACgizNNUxUVFerTp49stubr6YR2SZs3b1Z+fn68mwEAAAAA6GY2bNigfv36NXuc0C4pLS1NkvXDSk9Pj3NrAAAAAABdXXl5ufLz8yN5tDmEdinSJT49PZ3QDgAAAADoMHsaos1EdAAAAAAAJChCOwAAAAAACYrQDgAAAABAgmJMOwAAAADsgWmaCgQCCgaD8W4KOgm73S6Hw7HXy4oT2gEAAABgN3w+n7Zs2aLq6up4NwWdTHJysvLy8uRyudp8DUI7AAAAADQjFAppzZo1stvt6tOnj1wu115XTtH1maYpn8+n7du3a82aNRo6dKhstraNTie0AwAAAEAzfD6fQqGQ8vPzlZycHO/moBNJSkqS0+nUunXr5PP55PF42nQdJqIDAAAAgD1oa5UU3Vssfm/4zQMAAAAAIEER2gEAAAAASFCEdgAAAABAq0ybNk2ZmZl7fR3DMDR9+vS9vk5XRmgHAAAAgG7okksu0emnnx7vZmAPCO0AAAAAACQoQjsAAAAAIMpjjz2mAw88UCkpKcrPz9fVV1+tysrKRudNnz5dQ4cOlcfj0cSJE7Vhw4ao42+99ZYOPvhgeTweDRo0SHfffbcCgUBHfYwugdAOAAAAAIhis9n01FNPaenSpXrxxRc1e/Zs/d///V/UOdXV1br//vv10ksvaf78+SotLdW5554bOf7JJ5/ooosu0nXXXacffvhBzz33nKZNm6b777+/oz9Op0ZoBwAAAABEmTJligoLCzVw4ECNHz9e9913n15//fWoc/x+v/70pz9pzJgxOuSQQ/Tiiy/qs88+01dffSVJuvvuu/W73/1OF198sQYNGqTjjz9e9957r5577rl4fKROyxHvBgAAAAAAEsvMmTP14IMP6scff1R5ebkCgYC8Xq+qq6uVnJwsSXI4HBo9enTkPfvuu68yMzO1bNkyHXbYYVq0aJHmz58fVVkPBoONroPdI7QDAAAAACLWrl2rU045RVdddZXuv/9+ZWVl6dNPP9Vll10mn8/X4rBdWVmpu+++W2eccUajYx6PJ9bN7rII7QAAAACAiAULFigUCunRRx+VzWaNqN61a7wkBQIBffPNNzrssMMkScuXL1dpaamGDx8uSTr44IO1fPlyDRkypOMa3wUR2gEAAACgmyorK9PChQuj9mVnZ8vv9+vpp5/Wqaeeqvnz5+vZZ59t9F6n06lrr71WTz31lBwOh6655hodccQRkRB/xx136JRTTlH//v111llnyWazadGiRVqyZInuu+++jvh4XQIT0QEAAABANzV37lyNGjUq6vHyyy/rscce00MPPaQDDjhAr7zyih588MFG701OTtbNN9+sX/7ylzrqqKOUmpqqf/3rX5HjEydO1DvvvKOPPvpIo0eP1hFHHKHHH39cAwYM6MiP2OkZpmma8W5EvJWXlysjI0NlZWVKT0+Pd3MAAAAAJAiv16s1a9aooKCAcdhotd39/rQ0h1JpBwAAAAAgQRHaAQAAAABIUIR2AAAAAAASFLPHAwAAAADirtYfVKjBlGs2w5DbaY9jixIDoR0AAAAAEFe1/qCWb6totH9Y77RuH9wJ7QAAAACAuApX2HPSPHLaDfmDpooqvFGV9+6K0A4AAAAASAhOuyG3wy4pGO+mJAwmogMAAAAAIEFRaQcAAACAVtpUWqOSKl+H3a9Hikt9M5M67H5IHIR2AAAAAGiFTaU1Ou7RufL6Qx12T4/Tplk3juuw4L527VoVFBTou+++08iRI1v0nmnTpmnKlCkqLS2NWTvWrV2r4cOGtqodXQ2hHQAAAABaoaTKJ68/pMmFQzokRG8qrdHUOStVUuVr9f02bNigO++8Ux988IF27NihvLw8nX766brjjjvUs2fPZt+Xn5+vLVu2KDs7u8X3Ouecc3TSSSe1qn3YM0I7AAAAALRB38wkFWSnxLsZzVq9erXGjBmjffbZR//85z9VUFCgpUuX6qabbtL777+vL774QllZWY3e5/P55HK5lJub26r7JSUlKSmJLvyxxkR0AAAAANAFTZ48WS6XSx999JHGjh2r/v37a9KkSZo5c6Y2bdqk2267TZI0cOBA3XvvvbrooouUnp6uK6+8UmvXrpVhGFq4cGHkem+//baGDh0qj8ejwsJCvfjiizIMI9Idftq0acrMzIycf9ddd2nkyJF6+eWXNXDgQGVkZOjcc89VRUX9euwffPCBjj76aOXlZOvYAwfpvLN+rtWrV3XEj6fTILQDAAAAQBdTXFysDz/8UFdffXWj6ndubq7OP/98/etf/5JZtw76I488ooMOOkjfffedbr/99kbXW7Nmjc466yydfvrpWrRokX79619HQv/urFq1StOnT9c777yjd955R/PmzdMf/vCHyPGqqirdcMMN+vSzL/SX196SzWbTL885S6FQx80XkOjoHg8AAAAAXcxPP/0k0zQ1fPjwJo8PHz5cJSUl2r59uyRp/PjxuvHGGyPH165dG3X+c889p2HDhunhhx+WJA0bNkxLlizR/fffv9t2hEIhTZs2TWlpaZKkCy+8ULNmzYq878wzz5Qk1fgCCqXn6qlnntOwgn5a/uMypfcZ1PoP3gVRaQcAAACALipcSd+TQw89dLfHly9frtGjR0ftO+yww/Z43YEDB0YCuyTl5eWpqKgo8vqnn37Seeedp/2G7aMjh/fXwQcMkyRt3LChRe3uDgjtAAAAANDFDBkyRIZhaNmyZU0eX7ZsmXr06KFevXpJklJS2mdCPafTGfXaMIyoru+nnnqqiouLNfXPz+ofb8/Qh3M+kST5fb52aU9nRGgHAAAAgC6mZ8+eOv744/XMM8+opqYm6tjWrVv1yiuv6JxzzpFhGC263rBhw/TNN99E7fv666/3qo07d+7U8uXL9fvf/16F48dr0NBhKi0p2atrdkWMaQcAAACANthUWrPnk+J4nz/96U868sgjNXHiRN13331RS7717dt3j+PRG/r1r3+txx57TDfffLMuu+wyLVy4UNOmTZOkFgf/XfXo0UM9e/bUX/7yF/Xo2Utffr9Cf3743jZdqysjtAMAAABAK/RIccnjtGnqnJUddk+P06YeKa5WvWfo0KH65ptvdOedd+rss89WcXGxcnNzdfrpp+vOO+9sco325hQUFOjNN9/UjTfeqCeffFJjxozRbbfdpquuukput7u1H0eSZLPZ9Nprr+k3v/mNDj14pAYMGqKHH3tcp006oU3X66oMs6UzE3Rh5eXlysjIUFlZmdLT0+PdHAAAAAAJwuv1as2aNSooKJDH44ns31Rao5Kqjht33SPFpb6ZSXs+sQPdf//9evbZZ7UhBpPG1fgC+qmoUn0zk+R22FUbCGpTaY2G5qQqydV5a83N/f5ILc+hnffTAwAAAECc9M1MSrgQ3d6eeeYZjR49Wj179tT8+fP18MMP65prrol3s7o8QjsAAAAAYI9++ukn3XfffSouLlb//v1144036pZbbol3s7o8QjsAAAAAYI8ef/xxPf744/FuRrfDkm8AAAAAACQoQjsAAAAAAAmK0A4AAAAAQIIitAMAAAAAkKAI7QAAAAAAJChmjwcAAACA1irdIFXv7Lj7JfeUMvM77n5IGIR2AAAAAGiN0g3S1NGSv6bj7ulMkiZ/TXDfCwMHDtSUKVM0ZcqUmF1z3LhxGjlypJ544omYXXNXhHYAAAAAaI3qnVZgP+ZGKaMDQnTZBumTR637tjC0P/vss7rppptUUlIih8OKfZWVlerRo4eOOuoozZ07N3Lu3LlzVVhYqJUrV2rw4MHNXjN8XklJiTIzM/fmE6EVCO0AAAAA0BYZ+VLPIfFuRZMKCwtVWVmpb775RkcccYQk6ZNPPlFubq6+/PJLeb1eeTweSdKcOXPUv3//3Qb2WDJNU8FgMPLHhPbm8/nkcrk65F7tgYnoAAAAAKCLGTZsmPLy8hpV1E877TQVFBToiy++iNpfWFiol19+WYceeqjS0tKUm5urX/7ylyoqKpIkrV27VoWFhZKkHj16yDAMXXLJJZKkUCikBx98UAUFBUpKStJBBx2kN998M+r6hmHo/fff1yGHHCK3261PP/1U48aN07XXXqspU6aoT+9eKhy1j1564e+qqqrStf/vCo3ZN18HDN9X77//fuRawWBQl112WeRew4YN05NPPhn12S+55BKdfvrpuv/++9WnTx8NGzasyZ/R3/72N2VmZmrWrFmSpCVLlmjSpElKTU1V7969deGFF2rHjh2R86uqqnTRRRcpNTVVeXl5evTRR9vwzbQeoR0AAAAAuqDCwkLNmTMn8nrOnDkaN26cxo4dG9lfU1OjL7/8UoWFhfL7/br33nu1aNEiTZ8+XWvXro0E8/z8fP373/+WJC1fvlxbtmyJhOUHH3xQL730kp599lktXbpU119/vS644ALNmzcvqj2/+93v9Ic//EHLli3TiBEjJEkvvviisrOz9fGnn+m8S67UTdf/Rhedf65GH36EXnt/rsYdd5wuvPBClZRVSLL+QNCvXz+98cYb+uGHH3THHXfo1ltv1euvvx51r1mzZmn58uWaMWOG3nnnnUY/mz/+8Y/63e9+p48++kjHHXecSktLNX78eI0aNUrffPONPvjgA23btk1nn3125D033XST5s2bp7feeksfffSR5s6dq2+//XZvvqIWoXs8AAAAAHRBhYWFmjJligKBgGpqavTdd99p7Nix8vv9evbZZyVJn3/+uWpra1VYWKj+/ftH3jto0CA99dRTGj16tCorK5WamqqsrCxJUk5OTmRMe21trR544AHNnDlTY8aMibz3008/1XPPPaexY8dGrnnPPffo+OOPj2rjQQcdpN///veq8QV02TXXa9qfn1DPnj11ya8u14aSap3//27Q83/9i96d94V+MWm83E6n7r777sj7CwoK9Pnnn+v111+PCtgpKSn629/+1mS3+Jtvvlkvv/yy5s2bp/3331+S9Kc//UmjRo3SAw88EDnv+eefV35+vlasWKE+ffro73//u/7xj3/ouOOOk2T9waFfv36t/2JaidAOAAAAAF3QuHHjVFVVpa+//lolJSXaZ5991KtXL40dO1aXXnqpvF6v5s6dq0GDBql///5asGCB7rrrLi1atEglJSUKhUKSpPXr12u//fZr8h4rV65UdXV1ozDu8/k0atSoqH2HHnpoo/eHK+6SZLfb1SMrS/vtf6CcdpvyeySrT8YASVLxju0KmaYkaerUqXr++ee1fv161dTUyOfzaeTIkVHXPfDAA5sM7I8++qiqqqr0zTffaNCgQZH9ixYt0pw5c5SamtroPatWrYrc5/DDD4/sz8rKarbrfSwR2gEAAACgCxoyZIj69eunOXPmqKSkJFL17tOnj/Lz8/XZZ59pzpw5Gj9+vKqqqjRx4kRNnDhRr7zyinr16qX169dr4sSJ8vl8zd6jsrJSkvTuu++qb9++UcfcbnfU65SUlEbvdzqdUa8Nw5DTacVUp90mp90a0R0KWYH9tdde029/+1s9+uijGjNmjNLS0vTwww/ryy+/3OO9JOmYY47Ru+++q9dff12/+93voj7HqaeeqoceeqjRe/Ly8rRy5comr9cR4jqm/cEHH9To0aOVlpamnJwcnX766Vq+fHnUOePGjZNhGFGP//f//l/UOevXr9fJJ5+s5ORk5eTk6KabblIgEOjIjwIAAAAACaewsFBz587V3LlzNW7cuMj+Y489Vu+//76++uorFRYW6scff9TOnTv1hz/8Qcccc4z23XffyCR0YeHKdTAYjOzbb7/95Ha7tX79eg0ZMiTqkZ8f++Xw5s+fryOPPFJXX321Ro0apSFDhmjVqlUtfv9hhx2m999/Xw888IAeeeSRyP6DDz5YS5cu1cCBAxt9jpSUFA0ePFhOpzPqjwMlJSVasWJFTD9fU+JaaZ83b54mT56s0aNHKxAI6NZbb9UJJ5ygH374IeovI1dccYXuueeeyOvk5OTIdjAY1Mknn6zc3Fx99tln2rJliy666CI5nc6o8QgAAAAAEFNlGxL+PoWFhZo8ebL8fn/U+PKxY8fqmmuukc/nU2FhoRwOh1wul55++mn9v//3/7RkyRLde++9UdcaMGCADMPQO++8o5NOOklJSUlKS0vTb3/7W11//fUKhUI6+uijVVZWpvnz5ys9PV0XX3xxm9velKFDh+qll17Shx9+qIKCAr388sv6+uuvVVBQ0OJrHHnkkXrvvfc0adIkORwOTZkyRZMnT9Zf//pXnXfeefq///s/ZWVlaeXKlXrttdf0t7/9Tampqbrssst00003qWfPnsrJydFtt90mm6396+BxDe0ffPBB1Otp06YpJydHCxYs0LHHHhvZn5ycrNzc3Cav8dFHH+mHH37QzJkz1bt3b40cOVL33nuvbr75Zt11111NjmOora1VbW1t5HV5eXmMPhEAAACALi+5p+RMkj7pmCW/JFn3S+7Z6rcVFhaqpqZG++67r3r37h3ZP3bsWFVUVESWhpOsPHbrrbfqqaee0sEHH6xHHnlEP/vZzyLv6du3r+6++2797ne/06WXXqqLLrpI06ZN07333qtevXrpwQcf1OrVq5WZmamDDz5Yt956695/7l38+te/1nfffadzzjlHhmHovPPO09VXXx21LFxLHH300Xr33Xd10kknyW6369prr9X8+fN1880364QTTlBtba0GDBigE088MRLMH3744Ug3+rS0NN14440qKyuL+WfclWGadaP5E8DKlSs1dOhQff/99zrggAMkWd3jly5dKtM0lZubq1NPPVW33357pNp+xx136O2339bChQsj11mzZo0GDRqkb7/9ttHkB5J01113Rc04GFZWVqb09PT2+XAAAAAAOh2v16s1a9aooKBAHo+n/kDpBql6Z8c1JLmnlBn77uaJosYX0E9FleqbmSS3wx51rDYQ1KbSGg3NSVWSq3NNy9bs74+s4nFGRsYec2jCfOJQKKQpU6boqKOOigR2SfrlL3+pAQMGqE+fPlq8eLFuvvlmLV++XP/5z38kSVu3bo36i5GkyOutW7c2ea9bbrlFN9xwQ+R1eXl5u4y3AAAAANBFZeZ36RCNxJEwoX3y5MlasmSJPv3006j9V155ZWT7wAMPVF5eno477jitWrVKgwcPbtO93G53o5kMAQAAAABINHGdPT7smmuu0TvvvKM5c+bscXH68Lp44Sn3c3NztW3btqhzwq+bGwcPAAAAAEBnENfQbpqmrrnmGv33v//V7NmzWzTjX3jseniyhDFjxuj777+PWo5gxowZSk9P13777dcu7QYAAAAAoCPEtXv85MmT9eqrr+qtt95SWlpaZAx6RkaGkpKStGrVKr366qs66aST1LNnTy1evFjXX3+9jj32WI0YMUKSdMIJJ2i//fbThRdeqD/+8Y/aunWrfv/732vy5Ml0gQcAAAAQEwk0fzc6kVj83sS10v7nP/9ZZWVlGjdunPLy8iKPf/3rX5Ikl8ulmTNn6oQTTtC+++6rG2+8UWeeeab+97//Ra5ht9v1zjvvyG63a8yYMbrgggt00UUXRa3rDgAAAABt4XQ6JUnV1dVxbgk6o/DvTfj3qC3iWmnf018d8vPzNW/evD1eZ8CAAXrvvfdi1SwAAAAAkGQVCTMzMyPDcZOTk2UYRpxb1fXU+gIyAz75am1SMLq27AuEZAZ8qvV6ZYQSZi713TJNU9XV1SoqKlJmZqbsdvue39SMzvGJAQAAACBOwhNcN5xHC7HlC4RUVFErf7JTDlv0H0UCIVMl1X6pwi2XIyHmUm+xzMzMvZ4gndAOAAAAALthGIby8vKUk5Mjv98f7+Z0SSu2Veiutxfo+gn7qG+PpKhjm0pq9PicFfrzBYeooHdanFrYek6nc68q7GGEdgAAAABoAbvdHpMQhsZsjlptqgjKJ4dkd0Ud88mvTRVB2RwueTyeOLUwfjpX3wIAAAAAALoRQjsAAAAAAAmK0A4AAAAAQIIitAMAAAAAkKAI7QAAAAAAJChCOwAAAAAACYrQDgAAAABAgiK0AwAAAACQoAjtAAAAAAAkKEI7AAAAAAAJitAOAAAAAECCIrQDAAAAAJCgCO0AAAAAACQoQjsAAAAAAAmK0A4AAAAAQIIitAMAAAAAkKAI7QAAAAAAJChCOwAAAAAACYrQDgAAAABAgiK0AwAAAACQoAjtAAAAAAAkKEI7AAAAAAAJitAOAAAAAECCIrQDAAAAAJCgCO0AAAAAACQoQjsAAAAAAAmK0A4AAAAAQIIitAMAAAAAkKAI7QAAAAAAJChCOwAAAAAACYrQDgAAAABAgiK0AwAAAACQoAjtAAAAAAAkKEI7AAAAAAAJitAOAAAAAECCIrQDAAAAAJCgCO0AAAAAACQoQjsAAAAAAAmK0A4AAAAAQIIitAMAAAAAkKAI7QAAAAAAJChCOwAAAAAACYrQDgAAAABAgiK0AwAAAACQoAjtAAAAAAAkKEI7AAAAAAAJyhHvBgAAAAAAsCcriyoj2yluhwqyU+LYmo5DaAcAAAAAJCyP0+ogPuVfC6P2z/ntuG4R3AntAAAAAICElZeRpMfOPkhef0iStKm0RlPnrFRVbSDOLesYhHYAAAAAQELLy0iKdxPihonoAAAAAABIUIR2AAAAAAASFKEdAAAAAIAERWgHAAAAACBBEdoBAAAAAEhQhHYAAAAAABIUoR0AAAAAgARFaAcAAAAAIEER2gEAAAAASFCEdgAAAAAAEhShHQAAAACABEVoBwAAAAAgQRHaAQAAAABIUIR2AAAAAAASFKEdAAAAAIAERWgHAAAAACBBEdoBAAAAAEhQhHYAAAAAABIUoR0AAAAAgARFaAcAAAAAIEER2gEAAAAASFCEdgAAAAAAEhShHQAAAACABEVoBwAAAAAgQRHaAQAAAABIUIR2AAAAAAASFKEdAAAAAIAERWgHAAAAACBBEdoBAAAAAEhQhHYAAAAAQEL4qahCoZAZ72YkFEI7AAAAACAhvDB/reav2hHvZiQUQjsAAAAAIGGs2FYR7yYkFEI7AAAAACBhbCiuiXcTEgqhHQAAAAAQV6ZZP459fXF11OvujtAOAAAAAIirGn8wantHZW0cW5NYHPFuAAAAAACge1qzo0pVtQF9u740av+64mr1SvPEp1EJhtAOAAAAAOhwa3ZUqfCRuU0eW7+zWocOyOrYBiUouscDAAAAADpcVW1AkjS5cIh+cUi/qGMbSqrj0aSERGgHAAAAAMRN38wk2W2GJCkzySnJmowOFkI7AAAAACCuSqr9kqSD8jMlSVvKvPIFQnFsUeIgtAMAAAAA4qqk2idJKshOUZrHIdOUNtJFXhKhHQAAAAAQZyVVVmjPSnapf1ayJLrIhxHaAQAAAABxFa6090hxRkL7BkK7JEI7AAAAACCOQqapkiprTHuPZJfyqbRHIbQDAAAAAOKm2hdU0DRlSMpIrq+0ryuulmma8W1cAohraH/wwQc1evRopaWlKScnR6effrqWL18edY7X69XkyZPVs2dPpaam6swzz9S2bduizlm/fr1OPvlkJScnKycnRzfddJMCgUBHfhQAAAAAQBuU11hV9vQkpxw2m/r1SJJhSBXegMrqjnVncQ3t8+bN0+TJk/XFF19oxowZ8vv9OuGEE1RVVRU55/rrr9f//vc/vfHGG5o3b542b96sM844I3I8GAzq5JNPls/n02effaYXX3xR06ZN0x133BGPjwQAAAAAaIVyb7hrvLVGu9thV166RxJd5CXJEc+bf/DBB1Gvp02bppycHC1YsEDHHnusysrK9Pe//12vvvqqxo8fL0l64YUXNHz4cH3xxRc64ogj9NFHH+mHH37QzJkz1bt3b40cOVL33nuvbr75Zt11111yuVzx+GgAAAAAgBaoqLF6SWel1Ge3/KxkbS7zan1xtUb0y4xTyxJDQo1pLysrkyRlZWVJkhYsWCC/368JEyZEztl3333Vv39/ff7555Kkzz//XAceeKB69+4dOWfixIkqLy/X0qVLm7xPbW2tysvLox4AAAAAgI5XX2mvD+0s+1YvYUJ7KBTSlClTdNRRR+mAAw6QJG3dulUul0uZmZlR5/bu3Vtbt26NnNMwsIePh4815cEHH1RGRkbkkZ+fH+NPAwAAAABoiXBozyS0NylhQvvkyZO1ZMkSvfbaa+1+r1tuuUVlZWWRx4YNG9r9ngAAAACAxsqb6B4fDu2bSmoUCIXi0q5EkRCh/ZprrtE777yjOXPmqF+/fpH9ubm58vl8Ki0tjTp/27Ztys3NjZyz62zy4dfhc3bldruVnp4e9QAAAAAAdLxdJ6KTpOw0t5KcdgVCpraUeuPVtIQQ19BumqauueYa/fe//9Xs2bNVUFAQdfyQQw6R0+nUrFmzIvuWL1+u9evXa8yYMZKkMWPG6Pvvv1dRUVHknBkzZig9PV377bdfx3wQAAAAAECblHutSnuPBpV2m2EoPytJkrShpHt3kY/r7PGTJ0/Wq6++qrfeektpaWmRMegZGRlKSkpSRkaGLrvsMt1www3KyspSenq6rr32Wo0ZM0ZHHHGEJOmEE07QfvvtpwsvvFB//OMftXXrVv3+97/X5MmT5Xa74/nxAAAAAAB7UFVb1z0+OXrlr/weyVqxrVLri6t15OB4tCwxxDW0//nPf5YkjRs3Lmr/Cy+8oEsuuUSS9Pjjj8tms+nMM89UbW2tJk6cqGeeeSZyrt1u1zvvvKOrrrpKY8aMUUpKii6++GLdc889HfUxAAAAAAB7wW4zlOqJjqf9eljj2rt79/i4hnbTNPd4jsfj0dSpUzV16tRmzxkwYIDee++9WDYNAAAAANBBeiQ7ZTOMqH0pbrskyesPxqNJCSMhJqIDAAAAAHRfPXbpGi9JbocV2msDzB4PAAAAAEDcNJyELsztsOKqN0ClHQAAAACAuNl1EjpJcjutuFrrp9IOAAAAAEDcNFyjPSzcPZ5KOwAAAAAAcdRU93iPg0q7RGgHAAAAAMRZkxPROcMT0QVbtPJYV0VoBwAAAADE1e4moguZUiBEaAcAAAAAoMPU+OrHqjc5pt1ZH1e7cxd5QjsAAAAAoMMVV/skSS6HTckuR6PjDptNdpshyeoi310R2gEAAAAAHW5nZa0kKd3TOLCHeZzhtdqptAMAAAAA0GGKq6xKe5qncdf4sPCyb7V+Ku0AAAAAAHSYnXWhPSOp+dAeWfaNSjsAAAAAAB2nvtLefPf4hsu+dVeEdgAAAABAhyuutEJ7+m67x9eNae/Gs8c3/ycNAAAAAADaSbh7fHq4e3z5JslfU3+CMykS2rtzpZ3QDgAAAADocDurGsweX75J+s+Vjc5x935AUvdep53QDgAAAADocOU1AUlSissh+cutnSPOllJypKoiafHrctussN6dl3wjtAMAAAAAOlQwZKqmbhk3j8tefyAlR8roG3npthuSunf3eCaiAwAAAAB0qEpvILIdXtatKR5HXWjvxt3jCe0AAAAAgA5V7vVHth325mOpu64IT6UdAAAAAIAO0jC07w6VdkI7AAAAAKCDhSeh2xN3OLR344noCO0AAAAAgA7V0kp7eCI6L93jAQAAAADoGBXeVlba6R4PAAAAAEDHKK+h0t5ShHYAAAAAQIdiIrqWI7QDAAAAADpUi7vH28MT0VFpBwAAAACgQ7S4e7zDemb2eAAAAAAAOkhrZ4+nezwAAAAAAB2ktbPHB01TgWD3DO6EdgAAAABAh2rtRHSS5O2mXeQJ7QAAAACADlVe07JKu8NmyG4Ld5HvnpPREdoBAAAAAB2qooWVdknyOKzY2l0noyO0AwAAAAA6jGmaKm/hmHZJcjvtkgjtAAAAAAC0u2pfUMGQ2eLz3XWVdi/d4wEAAAAAaF/hmeNtxh5OrOOOdI8ntAMAAAAA0K7CM8enuh0tOt/tqOse303Xaie0AwAAAAA6THmNFdpTWhranXXd4xnTDgAAAABA+wp3j0922Vt0vidcaad7PAAAAAAA7SvcPb7FlfbwmHa6xwMAAAAA0L7C3eNbPKbdyUR0AAAAAAB0iPJWdo+PTETHmHYAAAAAANpXa7vHe8IT0XXT7vEt+ykBAAAAABAD5TVWpT3SPb50g2Tb0uz57m4+ER2hHQAAAADQYSrqKu3JoUprxycPS8ZWa9vhbnR+/Zh2Ku0AAAAAALSr8Jj2VJsV3jX0eKlXhhXYU7IbnR+utHv9VNoBAAAAAGhX4dnjU5yGtcOTJWXkNnu+hyXfAAAAAADoGJHu8eHQvgcs+QYAAAAAQAeJdI93tSyOsuQbAAAAAAAdpFH3+D1wO7r3RHSEdgAAAABAh6gNBCPhO8XV0u7xdZX2bjoRHaEdAAAAANAhKuq6xktSkqNloT08EZ2XSjsAAAAAAO0n3DU+ze2Q3dbS7vFWpT0YMhUIdb/gTmgHAAAAAHSIcKU9zdPy1cc9zvrY2h2XfSO0AwAAAAA6xIptFZIkl8OmlcWBPZxtcdhtshtWVb47TkZHaAcAAAAAtLs1O6p005uLJUlrd1ZrykclkiSPw9zjeyNrtXfDyegI7QAAAACAdldVW19Z3zc3TQ8UZuoxx1TlJe25eu7uxpPRtXwgAQAAAAAAMZCd6lZBD0lGcYvOtyaj86vWH4wsAdddUGkHAAAAAHSoZFfrgne4e3x3rLQT2gEAAAAAHSrZ1bpO3566Zd9qA4xpBwAAAACgXaW4W1lpd4QnoqPSDgAAAABAu2ptpT0yezyVdgAAAAAA2lerx7RHusdTaQcAAAAAoF21PrTXTURH93gAAAAAANpXiruVE9E5mYgOAAAAAIAOkdzKtdbrx7RTaQcAAAAAIOaCITOyndzKSnt4TLvXT6UdAAAAAICYq/HVB+7Wjmn3OKi0AwAAAADQbqp8AUmSw2bIaW9dFI10j6fSDgAAAABA7FXVWqE9qZXj2SWWfAMAAAAAoF1V1oV2T5tCO93jAQAAAABoN9V1Y9o9ztbHUHd4yTe6xwMAAAAAEHtVMai0e6m0AwAAAAAQe3vTPd4TGdNOpR0AAAAAgJir7x7fhtAemT2eSjsAAAAAADFXGZk9vg1j2usq7YGQqWDIjGm7Eh2hHQAAAADQ7vZqTHuDoO8Pdq9qO6EdAAAAANDu9qZ7vMNmyGZY291t2TdHvBsAAAAAAOj62jQRXdkGSZLhTJLbYVeNPyg/oR0AAAAAgNiqas2Ydofbev74kcgut/1O1fglXzfrHk9oBwAAAAC0u1Z1j0/Jlo65UQrUSlVF0uLX5al7m49KOwAAAAAAsdXqiehSsqNeuh3WoPbuVmlnIjoAAAAAQLsyTbNtY9obcNvrQns3q7QT2gEAAAAA7arGH1R4eXVPG9Zpl+or7Sz5BgAAAABADJXXBCLbLnsbQ7ud7vEAAAAAAMRcudcf2TYMo03X8DjoHg8AAAAAQMxVNAjtbcVEdAAAAAAAtIOG3ePbyl03f52fSjsAAAAAALFTHoNKe7h7fC2hHQAAAACA2Cn3xqLSzuzxAAAAAADEXHlNDMe0U2kHAAAAACB2YtE9niXfAAAAAABoBxWx6B4fqbSbe32tzoTQDgAAAABoVzHpHs+YdgAAAAAAYi8WE9F5HNYzY9oBAAAAAIihCsa0txmhHQAAAADQrpg9vu0I7QAAAACAdhWL7vGuukp7IERoBwAAAAAgZmLRPd5pqwvtQWaP7zAff/yxTj31VPXp00eGYWj69OlRxy+55BIZhhH1OPHEE6POKS4u1vnnn6/09HRlZmbqsssuU2VlZQd+CgAAAABAc2oDQXn9e18dd9mt5+4V2eMc2quqqnTQQQdp6tSpzZ5z4oknasuWLZHHP//5z6jj559/vpYuXaoZM2bonXfe0ccff6wrr7yyvZsOAAAAAGiBWKzRLknOuu7x3Y0jnjefNGmSJk2atNtz3G63cnNzmzy2bNkyffDBB/r666916KGHSpKefvppnXTSSXrkkUfUp0+fmLcZAAAAANBy4dCe5LSrxh9s83Wc3XRwd8J/7Llz5yonJ0fDhg3TVVddpZ07d0aOff7558rMzIwEdkmaMGGCbDabvvzyy2avWVtbq/Ly8qgHAAAAACD2wjPHp7r3rmZsGEa3rLYndGg/8cQT9dJLL2nWrFl66KGHNG/ePE2aNEnBoPXXma1btyonJyfqPQ6HQ1lZWdq6dWuz133wwQeVkZEReeTn57fr5wAAAACA7qq8bhK6FLd9r6/ltCd0hG0Xce0evyfnnntuZPvAAw/UiBEjNHjwYM2dO1fHHXdcm697yy236IYbboi8Li8vJ7gDAAAAQDsId49Pdu19/LRCe9u72HdGnerPFIMGDVJ2drZWrlwpScrNzVVRUVHUOYFAQMXFxc2Og5escfLp6elRDwAAAABA7IW7x8ei0u7qhpX2TvWJN27cqJ07dyovL0+SNGbMGJWWlmrBggWRc2bPnq1QKKTDDz88Xs0EAAAAANQJV9r3dky7JDkdjGlvkUGDBkVNCBdWWlqqQYMGtfg6lZWVWrhwoRYuXChJWrNmjRYuXKj169ersrJSN910k7744gutXbtWs2bN0mmnnaYhQ4Zo4sSJkqThw4frxBNP1BVXXKGvvvpK8+fP1zXXXKNzzz2XmeMBAAAAIAGEx7THrnt899KmT7x27drIZHAN1dbWatOmTS2+zjfffKNRo0Zp1KhRkqQbbrhBo0aN0h133CG73a7FixfrZz/7mfbZZx9ddtllOuSQQ/TJJ5/I7XZHrvHKK69o33331XHHHaeTTjpJRx99tP7yl7+05WMBAAAAAGIsVrPHS92ze3yrfmpvv/12ZPvDDz9URkZG5HUwGNSsWbM0cODAFl9v3LhxMk2z2eMffvjhHq+RlZWlV199tcX3BAAAAAB0nHD3eGaPb5tWhfbTTz9dkrU+3sUXXxx1zOl0auDAgXr00Udj1jgAAAAAQOcWy+7xLgehfbdCoZAkqaCgQF9//bWys7PbpVEAAAAAgK6hvCaGE9HZu99EdG36qa1ZsybW7QAAAAAAdEHhSjvd49umzX/qmDVrlmbNmqWioqJIBT7s+eef3+uGAQAAAAA6v/CYdmaPb5s2/dTuvvtu3XPPPTr00EOVl5cnw+h+XRQAAAAAAHsWy9njCe0t9Oyzz2ratGm68MILY90eAAAAAEAXEQqZqvTFbvb47jgRXZs+sc/n05FHHhnrtgAAAAAAupCK2oDCq3ynxGSd9u7Xy7tNof3yyy9nbXQAAAAAwG6Fu8a7HbaYdG2ne3wLeb1e/eUvf9HMmTM1YsQIOZ3OqOOPPfZYTBoHAAAAAOi8wpPQpSc593BmyxDaW2jx4sUaOXKkJGnJkiVRx5iUDgAAAAAg1S/3lubZ+67xEqG9xebMmRPrdgAAAAAAuphw9/h0T4wq7Y7uVyTufn+mAAAAAAB0iFh3j3dRaW+ZwsLC3XaDnz17dpsbBAAAAADoGmLdPZ7Q3kLh8exhfr9fCxcu1JIlS3TxxRfHol0AAAAAgE6uvKau0h6r7vGE9pZ5/PHHm9x/1113qbKycq8aBAAAAADoGirqKu3pSTGaiM7R/UJ7TD/xBRdcoOeffz6WlwQAAAAAdFLh7vGxq7QzEd1e+fzzz+XxeGJ5SQAAAABAJ1XfPZ4x7W3Vpp/cGWecEfXaNE1t2bJF33zzjW6//faYNAwAAAAA0LlV1Ia7xzOmva3aFNozMjKiXttsNg0bNkz33HOPTjjhhJg0DAAAAADQucV6IjpXNxzT3qbQ/sILL8S6HQAAAACALibWS75RaW+lBQsWaNmyZZKk/fffX6NGjYpJowAAAAAAnV+Ft67SnuSULxDa6+s1nIguGDL3+nqdQZtCe1FRkc4991zNnTtXmZmZkqTS0lIVFhbqtddeU69evWLZRgAAAABAJ7FmR5WqagMyTVNlNfWzx++orN3razestPuDe/9HgM6gTX0Lrr32WlVUVGjp0qUqLi5WcXGxlixZovLycv3mN7+JdRsBAAAAAJ3Amh1VKnxkrk55+lOd+qf5kWp4rLrHN5w9PhaV+86gTT+5Dz74QDNnztTw4cMj+/bbbz9NnTqViegAAAAAoJuqqrW6w08uHKKq2oCmfbZWNkNKdtljcn2bzZDdMBQ0TfmotDcvFArJ6Ww8+5/T6VQo1D1+cAAAAACApvXNTFJG3TJvqW6HDMPYwztazlE3rr27VNrbFNrHjx+v6667Tps3b47s27Rpk66//nodd9xxMWscAAAAAKBz8vqDkqRkV2y6xoc5bHWhnUp78/70pz+pvLxcAwcO1ODBgzV48GAVFBSovLxcTz/9dKzbCAAAAADoZLx+K1SnuGPTNT7MUTeu3d9NKu1t+pNHfn6+vv32W82cOVM//vijJGn48OGaMGFCTBsHAAAAAOicwpX2FHdsK+3hZd+otDdh9uzZ2m+//VReXi7DMHT88cfr2muv1bXXXqvRo0dr//331yeffNJebQUAAAAAdBI14dAe8+7xVoxlTHsTnnjiCV1xxRVKT09vdCwjI0O//vWv9dhjj8WscQAAAACAzqk2UmmPdfd4Ku3NWrRokU488cRmj59wwglasGDBXjcKAAAAANC51UTGtLdPpd0fMGN63UTVqtC+bdu2Jpd6C3M4HNq+ffteNwoAAAAA0Ll526l7PGPad6Nv375asmRJs8cXL16svLy8vW4UAAAAAKBz8wbaqXu8jXXam3XSSSfp9ttvl9frbXSspqZGd955p0455ZSYNQ4AAAAA0DmFK+2pse4eH17yrZtU2lv10/v973+v//znP9pnn310zTXXaNiwYZKkH3/8UVOnTlUwGNRtt93WLg0FAAAAAHQeNT4rtCe3U/f42m5SaW/VT69379767LPPdNVVV+mWW26RaVoD/w3D0MSJEzV16lT17t27XRoKAAAAAOg8vIHwRHSx7h5PpX23BgwYoPfee08lJSVauXKlTNPU0KFD1aNHj/ZoHwAAAACgE2q/7vHda0x7m396PXr00OjRo2PZFgAAAABAFxEO7bHuHh+utHeX2eNj+9MDAAAAAHR7gVBI/qA1nLpRpb10g2Q4pLINbbp2eEw73eMBAAAAAGgDr78+UCe56sa0l220nj95WDK21p/scLfq2uHZ4+keDwAAAABAG4S7xkuSvW5ddfmqrOehx0u9Mqxth1tKyW7VtbvbOu2EdgAAAABATDUM7Y14sqSM3DZfOzIRXTfpHm+LdwMAAAAAAF1Lw+7xseYML/kWMNvtHomE0A4AAAAAiKma3VXa9xKVdgAAAAAA9kJtO4Z2p717LflGaAcAAAAAxFS1rx0r7d1sIjpCOwAAAAAgpto1tNvDY9oJ7QAAAAAAtFq1L9Bu145U2ukeDwAAAABA67VvpZ3QDgAAAABAm7VnaHfSPR4AAAAAgLar8bd/9/haQjsAAAAAAK3XERPRBUKmQiGz3e6TKAjtAAAAAICYaklod3m3K7lidauv7ayrtEvdY1w7oR0AAAAAEFPBPVTA7YFqHfjVzRrxxW+VUr6qVdcOV9olqdZPaAcAAAAAoNXsDSriu+q7+nW5aotlKKQ+a//d5uvWBtqvG36iILQDAAAAAGIu2WVvcn+ab5vy1r8Ted1z2xfyVG1u0z26w2R0hHYAAAAAQMwlOZsO7aO2/Vs2M6CSngerJPtQq9q+bnqb7uH1U2kHAAAAAKDVkl2ORvuOtS1Sv8rFChl2rR32K20a+HNJUq/Ns+WsLW71Pai0AwAAAADQBuHu8SuLKrVkU5lW7fDqdsc/JElb80+WN6WfKjL3U3nGvrKZgagu800q2yDtXCmVbojs6g5j2hv/6QMAAAAAgL2U5rHi5pR/LZQkXWp/X6c5N6nWnqqNg862TjIMbR54htIXPaDeGz/QpoFnKuhMib6Qw209f/yI9WzmSrpCUveYPZ7QDgAAAACIuZw0tx47+yB5/SE5/JU6fc5/paC0Mf9UBZ2pkfNKeh2q6pR8JVdtUO+NH2pzwRnRF0rJlo65UQrUWq+3l0nLrE26xwMAAAAA0AapHqfyMpJUkJ2iA0M/yBWslNd0qqjXkdEnGjZtrhvbnrf+fzKCvsYXS8mWMvpaD09WZDcT0QEAAAAA0AZp7vqO3WlF30qSys1kyWgcQ3fkHqNaT7ZcvhL1LPq8xfeg0g4AAAAAQBukNgzt263QXqGkJs81bU4V5xwhSUopX9Xie3SHiegI7QAAAACAmEutm4hOoYBSdyyUJFWYTYd2SapOyZckJVVvavE9qLQDAAAAANAG4Up7culy2YM1CjiSVSN3s+d7k/tKkpKqNrb4Ht1h9nhCOwAAAAAgJkKmGdkOL/mWtn2BJKkycz9JRrPvrUnpJ0ly1xTJFqxt0f2YiA4AAAAAgBaqqg1EtlPdDql8k9I2zpMkVST12e17/a4MBRypMmTKU72lRfejezwAAAAAAC1U4bVCu8thk6Nqi/SfK5W26VPr2NrvrJMczXSRNwzVpLSui3x3mIjOsedTAAAAAADYs3BoT3bZJX+NnPLLY/hlylDlYddJ7kxrzfVm1KT0U1rZciVVtWwyuu5QaSe0AwAAAABiotzrlyQlO+2SpDTVSJKqUwcomDVkj+8Pj2tvcaWdiegAAAAAAGiZykilvW4SOsMK7RUZw1r0/kj3+Gq6x4cR2gEAAAAAMRGutCe56irt4dCeuW+L3l+TbFXaPVWbJHPPVXQvlXYAAAAAAFqm4Zh2I+hTirySpMoWhnZvUm+FDIfsIZ/c3u17PJ9KOwAAAAAALdQwtKeU/ySbYcrvSJU3KbdlF7DZ5U22loZryWR03WEiOkI7AAAAACAmKsIT0bkcSitZZu1LHSQZRouvER7X7mnBZHSEdgAAAAAAWqhhpT2t5AdrX2pBq65RP4N8SyrtdI8HAAAAAKBFGi75llZaF9rTBrXqGuHQntyCSjsT0QEAAAAA0EKVtValPcsslqu2WCFTqkrp36prRLrHt2DZNyrtAAAAAAC0UHmNFdr7eZdLkqrkUcjmatU1apKt0O7ylcnhr9jtubVU2gEAAAAA2DNfIKQav1X57ldTF9pNT6uvE3IkqdbdU1Ldeu27wUR0AAAAAAC0QGmNL7KdV22F9mq1PrRLDSej230XebrHAwAAAADQAmXV/sh2z8oVkqRq092ma7U8tIdkmmab7tFZENoBAAAAAHutpC60GwopxbtVklSttoZ2a1z7npZ9M03JF+zaXeQJ7QAAAACAvVZSbXWPT1atJKnWk62g7G26Vksr7VLXH9dOaAcAAAAA7LVw9/gMVUmSqtMGtvla4dDuqdkmI+Tf7bldfQZ5QjsAAAAAYK+FK+3ZRqkkqSZ1YJuv5Xf1UMCRLEMheaq3NHmOq66I39UnoyO0AwAAAAD2WnhMe19jh6S9q7TLMCLrtTfXRd5lNyTRPR4AAAAAgD0qrau0Fxh1k9ClFezV9fY0rt1ls0K710+lHQAAAACA3Sqtq7TnGcUKyaaa1P57db360N70DPLOSPd4Ku0AAAAAAOxWeEx7plGpypT+CtnbttxbWG1SjiTJVbujyeOR7vFMRAcAAAAAwO6FK+09VKnS1CF7fT2fO0uS5PIWN3m8fkw73eMBAAAAANit0pr6SntZWgxDe22xZJqNjjvr0izd4wEAAAAA2A3TNCOzx2calSpNHbrX1/TXhXZ7qFb2QHWj48weDwAAAABAC9T4g/LVheceqohJpT1kdyvgSJFUV23fRTi0M3s8AAAAAAC7Ea6yu+SX3Qypwm+Tyjbs9XWjusjvwtVNZo93xLsBAAAAAIDOLbxGe4YqtUp9ZH76mFS3XrscbZ9F3ufOUnLVBjmbCO3OyOzxXbvSTmgHAAAAAOyVyMzxRqV+NPOlocdLvTKswJ6S3ebr7rbSbuseY9oJ7QAAAACAvRJZo12VWhHK1wBPlpSRu9fX3W1or0uzXT20x3VM+8cff6xTTz1Vffr0kWEYmj59etRx0zR1xx13KC8vT0lJSZowYYJ++umnqHOKi4t1/vnnKz09XZmZmbrssstUWVnZgZ8CAAAAALq3hjPHLzf7xey6/kho39noWKTS3sW7x8c1tFdVVemggw7S1KlTmzz+xz/+UU899ZSeffZZffnll0pJSdHEiRPl9Xoj55x//vlaunSpZsyYoXfeeUcff/yxrrzyyo76CAAAAADQ7ZVV1kiyuscvD/WP2XV9np6SJFdtSaNj3WXJt7h2j580aZImTZrU5DHTNPXEE0/o97//vU477TRJ0ksvvaTevXtr+vTpOvfcc7Vs2TJ98MEH+vrrr3XooYdKkp5++mmddNJJeuSRR9SnT58O+ywAAAAA0F2VFG+XJGXYarUlmCWpIibX3V33eGdk9ngq7XGxZs0abd26VRMmTIjsy8jI0OGHH67PP/9ckvT5558rMzMzEtglacKECbLZbPryyy+bvXZtba3Ky8ujHgAAAACAtikpsUJ1isclyYjZdX3uHpIkZ22JZEZX1COVdn/XrrQnbGjfutVaHqB3795R+3v37h05tnXrVuXk5EQddzgcysrKipzTlAcffFAZGRmRR35+foxbDwAAAADdx87yKklSRkpSTK/rd/WQKUM2MyCHP7rY6uwms8cnbGhvT7fccovKysoijw0bNsS7SQAAAADQaW2vsrqop6enx/S6ps0hvytDUuMu8m66x8dXbq61PMC2bdui9m/bti1yLDc3V0VFRVHHA4GAiouLI+c0xe12Kz09PeoBAAAAAGib7T6nJCkts1fMrx0Z1+6NDu3Ouu7xXrrHx0dBQYFyc3M1a9asyL7y8nJ9+eWXGjNmjCRpzJgxKi0t1YIFCyLnzJ49W6FQSIcffniHtxkAAAAAupugt0o7QymSpOSsvJhfv37Zt+gZ5Otnj+/alfa4zh5fWVmplStXRl6vWbNGCxcuVFZWlvr3768pU6bovvvu09ChQ1VQUKDbb79dffr00emnny5JGj58uE488URdccUVevbZZ+X3+3XNNdfo3HPPZeZ4AAAAAOgAOzcuV0g22RRSalpPSY3XVN8b4cnodl2r3RXpHt+1K+1xDe3ffPONCgsLI69vuOEGSdLFF1+sadOm6f/+7/9UVVWlK6+8UqWlpTr66KP1wQcfyOPxRN7zyiuv6JprrtFxxx0nm82mM888U0899VSHfxYAAAAA6I62b1wlKVk97TWy22PfmdvnDq/Vvkv3eFv3mD0+rqF93LhxMk2z2eOGYeiee+7RPffc0+w5WVlZevXVV9ujeQAAAACAPdi+daOkfdTL0z7hOTym3bnrRHSO7tE9PmHHtAMAAAAAEl/Rzh2SpF6pzna5fmQiukaVduuZiegAAAAAAGjG9tJKSVKvjNR2uX5zob27TERHaAcAAAAAtI2/RturrUp3r+zs9rlFuHu8r0xGKBDZXx/aqbQDAAAAANDYjp+03cyQJOVkZbXLLfyudIUMuwyZcvpKI/ud3WT2eEI7AAAAAKBttv+o7WamJKlXurt97mHY5I8s+1bfRd5dV2kPhkwFgl03uBPaAQAAAABts/1HbVemJKlXajuFdjU9g3x4yTepa1fbCe0AAAAAgLYp+jHSPb5XWvuH9oaV9nD3eEny+rvuZHSEdgAAAABAm1RvW6VKJUvq+NBuMwy57FakpdIOAAAAAEBDfq92lJRIkpKcNqW6He12q+aWfXM7CO0AAAAAADS28ydtN9MlSb3SPDIMYw9vaDt/c6G9ro883eMBAAAAAGio6EcVhWeOb8eu8VKDSrs3OrQnuaxIS2gHAAAAAKChhsu9tePM8ZLkc/eU1LjS7nFYlfYaQjsAAAAAAA00DO3tXmm31ml3BCplD/ki+5NcdI8HAAAAAKCx7T9qu6zl3nLaObQHHSkK2lySJE+gLLLfUzemvcbHRHQAAAAAAFj8Xql4dYdV2mUYkcnokgOlkd1JTrrHAwAAAAAQbedKyQypyLDGmrd7aFf9uPakBpX27hDa228hPQAAAABA17T9R+vJsKrfvbzrpM1bpNL1kpLa5ZbhGeST/KWRfZEx7T5COwAAAAAAlu0/KmQa2hFIkST1mn6OZBRLoYGSHpAczpjfMjwZXVITY9q78kR0hHYAAAAAQOsULVOpUhWoG3GdffSlUo98qcQhzZLk6RHzW/qaGNPucVr3p3s8AAAAAABh25dru2nNHJ+lcjl75Es9h0ihoKSqdrllpHs8E9EBAAAAANAMf41UvEpF4ZnjjdIOua3P0/xEdF25ezyhHQAAAADQYpt+WiiZIW119JUkpaqmQ+7rd9WNafc3CO2u8DrthHYAAAAAQDe3ZkeVHvvHdEnSd7VWaF9gDtOaCnu739vvzpQkOc1aeVQrqX4iOrrHAwAAAAC6varagPa1rZckeTMK6vcHjHa/d9CepJDNmpU+2yiX1HBMe6jd7x8vhHYAAAAAQIsNMzZIksqcuR17Y8OQ35UpScqW1UU+sk47lXYAAAAAAKR9bVZo3+6L/Vrse+J3WTPW9zSs0B5e8o3QDgAAAADo9uzbvldO3WzxO0vrJ4STw90h949U2uu6x0fGtDMRHQAAAACgu/PsXCpJqnBkqdTRs/5ASk6H3D9SaQ93j2ciOgAAAAAALJ7yNZKkIvdAVQU6Pk6GK+29DMa0AwAAAAAQxVNmhfa1zkGSJHv7TxofpX5M+y6zx9M9HgAAAADQ3bnrKu3r7QMkSWmujr1/OLRnN9E93jTNjm1MByG0AwAAAAD2LBSUp2KdJGmzzVruLdXVsaX2cPf4cKXdU9c9PmRKvmDXXKud0A4AAAAA2LOStbIFvfKaTm03e0iS0pwdHdqju8d7HPbIMa+f0A4AAAAA6K62WTPHrzD7qaJuErq0OFXas1QhhYJy2g3ZbVYbuupkdIR2AAAAAMCeFf0gSVoeyo+E9tQOH9OeJlOGbIYpu69chmF0+cnoCO0AAAAAgD3btkSS9KOZrwq/Vd3u6Eq7DLtq7amSJEdtiSTJ08XXaie0AwAAAAD2bJtVaf/R7K/KcGjv4DHtkuR1pEmSHLWlkqQklxVru2pod8S7AQAAAACABOerlopXS5KWh/or4A93j49DaLeHQ7tVaQ93j/d20e7xhHYAAAAAwO5tXybJVMCVqR3eDNn81proGe54VNrTJTWotNM9HgAAAADQrdV1jfdmFEiSQjJkN6S0Dp6ITmpcaQ+PaWfJNwAAAABA97T+M0mS150d2ZWdZMhmxL/SzkR0AAAAAIDua+cqaeGrkiTv+m8ju3sld3xglxpORBc9pp3QDgAAAADofmorIpvekb+KbGcnxSm013WPt0dmj+/aE9ER2gEAAAAAzasurtswVJs3OrK7V3J84mSto+kx7VTaAQAAAADdz44V1nNKL5l2d2R3rzhX2h21pZJp0j0eAAAAANCNhUN7Rn7U7uy4jWm3JqKzhfxSbbmSXFasraF7PAAAAACg29mx3HrO7KeQaUZ2x6vSHrS5VGl6rBdVOyKV9toAoR0AAAAA0N1sr6+0F3ut0G4zpB6e+IR2SdphZlgblUX1Y9qptAMAAAAAupWqHVJVkbWd3k9F1VZoz3AZstviF9p3yuoir6rtTEQHAAAAAOimNi+s33Z6IqE9M45VdqlBpb2qqMFEdKE4tqj9ENoBAAAAAE3b8l3Uy6IqKxjHs2u8JO00w5X2HazTDgAAAADophpW2iVtC1fa3U2c24G2q35MO0u+AQAAAAC6py2Lo16Gu8cnTqWdMe0AAAAAgO6oulgqWx+1q6i6rnu8O1HGtG+v7x5PaAcAAAAAdBub68azZ/STJIVMJcxEdDsbhHaP04q1hHYAAAAAQPexZaH1nL2PJGlHrU3hCdozXHGutIeXfKvcXj+mnYnoAAAAAADdRngSuuxhkqSNVfXxMZ5rtEsNusfXlinJZoX1Gn9QpmnGsVXtg9AOAAAAAGgsXGnvZVXaN1bb49eWXZQpRaZhtcfjK5Fkdd/3BbveWu2EdgAAAABAtOpiqbRuErq67vGbqhIntEuGAu5MSVKSb2dkr9dHaAcAAAAAdHXhKnvWIMmVKknaWJ1Y8THg7iFJcnp3ylHXXb8rLvuWWD91AAAAAED8hcez542M7NqYUJV2RSrtqiyqn4yO0A4AAAAA6PLClfa8gyK7NiXQmHapvtKuqu1yO7vuWu2OeDcAAAAAAJBgwpX2PiMlSaa5+4noVpaGop47QqTSXrVdSS6rHt0VK+2EdgAAAABAvepiqXSdtZ13kFSyTjuVLm/QkCGp4aJqnrpEOWV2TdQlPB2QNBtW2sPd471dcK12QjsAAAAAoN6WRdZzj4FSUg+pZJ02mdmSpCyPoZ3e+tiel2rTY4UeeQP1b/c4rP3tLdhNxrQT2gEAAAAA9TYtsJ77jIrs2mj2kiT1TokO7VLHBPSmrPelq5+kmtJtkdnaumJoZyI6AAAAAEC9DV9Zz/mHR3aFQ3uvZCMeLYricVh/NLh/gRVnK3Zu1qKNZZKkmi7YPZ7QDgAAAACwhELShi+t7QahPdw9vndy/CNkXlJIjzmm6vIj8yVJ2UaFwiPtmT0eAAAAANB17VgheUslZ7KUe2Bk98a60J6TAJV2ScozimX06ilJsikol/zyySWvv+Nmr+8o8f8zCQAAAAAgMYSr7H0PkezOyO5w9/hECe2SZNoc8rsyJUke+SQxph0AAAAA0JVFusYfFtllmmZCdY9vyJ9ktStFtZII7QAAAACAriwS2o+I7Cr1hlSlJEmJMRFdQ36PFdpTZa0Tz0R0AAAAAICuqWqHtHOltZ0/OrJ7XZkVhHM8QbnsCRba6yrt6UaVpK45ER2hHQAAAAAg/fCW9dxjoFRdHNm9uiQgSRqUlniBOFxpz6gL7XSPBwAAAAB0PTtXSe/eYG2XrJWePtjaJ2l1aeKGdl9daM9UhSS6xwMAAAAAuqLaivrtfSZG7Vtd4pckDUoNdHSr9sifZM1q39Ow2uoNsOQbAAAAAKCrCfrqt/scGnUo3D1+cAJW2sNj2nsa5ZIkL5V2AAAAAECXs2OF9exKlVKyI7tDIVNr6rrHFyRiaK/rHp+jEkmMaQcAAAAAdEVbl1jPPQZKRv0M8ZvLalQblJwKqF9y4gXicPf4HKNUEqEdAAAAANAVbasL7VmDonav3m7Nyj7A2CpHAqZHv6enJCnV8EpiIjoAAAAAQFdjmvWhvcfA+v07Vmj1quWSpEHGlo5vVwuYNqe8zkwlqVYS67QDAAAAALqa4tVSjTUmXBn5kjPJ2v7PFVr96RuS6kJ7eH+C8bp7KknWRHpdsXu8I94NAAAAAADE0Yav6rftTim9r/Tz5yR/jVZ/nCkVSWn7n6AltT21sjTxllTzunrKY+yQZIV20zRlNBiX39kR2gEAAACgO1v/eeN96X0lSaurKySZenixRw8vrooc9iRQkvS6eyqrrtJumpIvGJLbYY9zq2IngX7UAAAAAIAOt+bjJndX+01trjQjryePcqlvqk0eh5SXmjgjrRt2j5ckr69rhfbE+UkDAAAAADpWyVqpZI1kNA65a8utrvBpLut131SbCjJtCRXYJanGlS2nEZRDVnu72rj2xPppAwAAAAA6zup51nPOfo0P1Y1f75tgIX1XXre17JvH5pdEaAcAAAAAdBWr51rPfQ9pfKgutPfrLKFddaG9i63Vntg/fQAAAABA+wiFpDV1lfZ+zYf2PmmJPRO715UtSUqSVxKVdgAAAABAV7BtiVS9U3KlNt09vswKv52l0p5s1livCe0AAAAAgE4v3DV+wFGSLXphMdM068e0pyV2bPS6eihkGvLUzSBPaAcAAAAAdH7h0D5oXKND26tNVfolmyH1SUns7vGmzakSpcqjWkl0jwcAAAAAdHaBWmndZ9Z2E6F9dVndJHRphpz2xA7tkrTDzFCSYVXamYgOAAAAANC5bfhKCtRIKTlSzvBGh8Nd4wdlNF6/PRFtNzOUVFdpp3s8AAAAAKBza9g13mhcSQ9X2gsyOkdk3KEMJdWNaad7PAAAAACgc9vNeHapQaU9s3NExu1mpjxG3Zh2XyjOrYmtzvENAAAAAABio6ZU2vyttT1obJOnhEP74E4S2neYVNoBAAAAAF3B2k8lMyT1HCpl9Gt02Bc0taGic1XadzCmHQAAAADQJeyha/z68pCCppTskHonJ/7M8ZK0XRnyGKzT3uHuuusuGYYR9dh3330jx71eryZPnqyePXsqNTVVZ555prZt2xbHFgMAAABAgls9x3pubjx7eBK6TJuMJiapS0Q7zAx56B4fH/vvv7+2bNkSeXz66aeRY9dff73+97//6Y033tC8efO0efNmnXHGGXFsLQAAAAAksB0/STtXSjanVHBMk6f8VGKF9iGdpGu8ZE1EF+4eX+MLxLk1seWIdwP2xOFwKDc3t9H+srIy/f3vf9err76q8ePHS5JeeOEFDR8+XF988YWOOOKIjm4qAAAAACS2H9+1ngceLXkymj5lp1WpHpbVOdZol6Ripclj+CVJNV5vnFsTWwn/p5OffvpJffr00aBBg3T++edr/fr1kqQFCxbI7/drwoQJkXP33Xdf9e/fX59//vlur1lbW6vy8vKoBwAAAAB0ecvfs573PbnRoTWhXC0pcWhRkRXah/dM+LgYEZRdbqdTkuT11sa5NbGV0N/C4YcfrmnTpumDDz7Qn//8Z61Zs0bHHHOMKioqtHXrVrlcLmVmZka9p3fv3tq6detur/vggw8qIyMj8sjPz2/HTwEAAAAACaCySNrwlbU9bFLUoTWlARX6HtMps7K0vsKUZE1E15m4PMmSpJpaX5xbElsJ/TVMmlT/izRixAgdfvjhGjBggF5//XUlJSW1+bq33HKLbrjhhsjr8vJygjsAAACArm3FB5JMKe8ga6m3nauk2gpJUlXRWklJOntAjV5fZ2WtZEfnmIQuzFkX2r1+xrTHTWZmpvbZZx+tXLlSxx9/vHw+n0pLS6Oq7du2bWtyDHxDbrdbbre7nVsLAAAAAAlk+fvW87CTrcD+9MH1x0IDJT0g01Y/jr2zzBwf5vCkSpJq/KE4tyS2Erp7/K4qKyu1atUq5eXl6ZBDDpHT6dSsWbMix5cvX67169drzJgxcWwlAAAAACQYX7W0qm6pt31PilTYdcyN0ilPSMf8VpK01df2Hs3x5kyyJtbzdq1Ce2JX2n/729/q1FNP1YABA7R582bdeeedstvtOu+885SRkaHLLrtMN9xwg7KyspSenq5rr71WY8aMYeZ4AAAAAGho9RwpUCNl9Jd6HyBtWWTtz8iXeg6RQkFJVdpabca1mXvDmZwuSaoJdq4eAnuS0KF948aNOu+887Rz50716tVLRx99tL744gv16tVLkvT444/LZrPpzDPPVG1trSZOnKhnnnkmzq0GAAAAgATzY92s8fmjrcC+Y0WTp22t6rxdyx2pWZKkmpBdpml2uu79zUno0P7aa6/t9rjH49HUqVM1derUDmoRAAAAAHQyoaC0vG599iX/th5hzuju8BWdeOJ1R0pPSZIpQ7WBkDzOzrPO/O4kdGgHAAAAAOylDV9JNSXW9lFTpB4DrW1nkpTet9HpPTyGSrydr5u8PbWnJOtzev1BQjsAAAAAoBMIV9klK7D3HLLb03NTrNC+stTqKh9+TnRmSo6c2i6/HKrx+pSZ7Ip3k2KC0A4AAAAAXZVp1o9nb6F+qYaW7ZSmzK6J2u9J8PQY9GTJI5/8cshbsVPKSo13k2IiwX/sAAAAAIA2K1omFa+SbA4p1LK10PbLtmvSIGfU0mkeh5SXmuArhtvs8hh+VZhSTdl2SQPi3aKYILQDAAAAQFf1/evWc/7h0rr5zZ4WNOvHsOen2RI/oDcjyR6SAlJNRXG8mxIznfObAAAAAADsXigkff+mtT1kwm5P3VplhXaHzRrT3lkl2a3P4a0oiXNLYofQDgAAAABd0YYvpLINkitNGnDUbk9dV2ZNNpeTbMhu67yh3eOw2l5TVR7nlsQO3eMBAAAAoCtaXNc1fvipksO921PX1IX23JROVNct22A9l9YPvk/q2U+qqlLNwN33LOhMCO0AAAAA0NUEfNIP063tEb/Y4+nryutCe3InqLKH/wDx8SPWs5kr6QqpbKOSkpIlVanG7DpRt+t8EgAAAACAZdUsqaZESu0tFYyVtn6/29PrK+2dILSnZEvH3CgFaq3X28ukZZJ8VfI47ZKkWn8wfu2LMUI7AAAAAHQ14a7xB5wp2ey7PbXab0YmouvdWbrHp2TXb1fUf75DBvSQzTDULys5Do1qH4R2AAAAAOhKaiuk5e9b2yPO3uPpK0pCCi/4lubqBJX23bj8mEHxbkLMEdoBAAAAoKvYuUr6/g0pUCNl9JdMSZsXSjtWNPuW5cVdpyt5V0RoBwAAAICuYOcq6emD61+XrZf+Oi76HGdSo7ct2xlq33ZhrxDaAQAAAKArqK2o2zAkmVLh76PHfjuTpPS+jd72/XYq7YmM0A4AAAAAXYop9Rom9T9ij2f6gqaW7CC0J7JOMjUgAAAAAGC3zAbd3Iec0KK3/FgcUm1QSnW2U5uw1wjtAAAAANAVbFpgPTs81trsLbBwm1VlH5a1+2XhED+EdgAAAADoCn5423ruN1pyelr0lu+KrNC+Tw+iYaLimwEAAACAzq58s7RuvrU94MgWv21hXWjfN4tomKiYiA4AAAAAOpOdq+pninenST0HS9++LJl1E8ql5bXoMiXekNaUWePg96F7fMIitAMAAABAZ7HrWuySNPkracG0Vl8qXGUflGFTmsuIQePQHugDAQAAAACdRbjCfsyN1kOSfvpIqtgseTJadanv6iahG5lDlT2REdoBAAAAoLPJyLceUv0EdMNOatUlwpX2Ub0J7YmM0A4AAAAAndnGr6zn4T9r8VtCphkJ7VTaExtj2gEAAACgsxt8nJTep8WnrykLqdwnuWxSMGRqbbnZjo3D3iC0AwAAAEBn5K+u3z7sila99aO1fkmSLySdPr3+Oh4SYsLhKwEAAACAzii8LnuPAmnoRGnr4ha/9fsia6m3kTk2nT3MJckK7HmpjKBONIR2AAAAAOhsgj5p9Txr+6DzJFvrwvbyYiu0D8+yqyCToJ7I+HYAAAAAoLPZ8JXkq7S2hxzXqrfW+E2tKbdCe346kTDR8Q0BAAAAQGezek79tq11Hai/3xFUqG7euQxXDNuEdkFoBwAAAIDOpnqn5Epp01vDS71JkmEYsWoR2gmhHQAAAAA6C7PB0mwFY9t0ie+2Bfd8EhIGoR0AAAAAOosNX1rPdrc04OhWv900TS0gtHcqzB4PAAAAAJ2BaUoL/2FtDzhSciVb2ztWRD/vxpIdIRVVm/LYJS/ZvVMgtAMAAABAZ7DiQ2nr99Z2wVjJmWRt/+eK6PPC+5swY61fknRwb7s+20xq7wwI7QAAAACQ6EJBaeZd9a+TMqX0vtLPn5P8NfX7nUnW/mbMWh+QJB2WR2jvLAjtAAAAAJDoFv1T2r5McqdJtRX1+3cT0He1uTKkpTtCMiQd2tshyRfzZiL2mIgOAAAAABKZv0aa84C1PfKCNl9m1jqryn5wb7syPSz11lkQ2gEAAAAgkX35rFS+ScrIl/b/eZsvEw7txw2gw3VnQmgHAAAAgERVXSx98ri1XXib5HC36TJVflOfbbJC+/EDCe2dCaEdAAAAABLVJ49KtWVS7wOkEWe3/TIbA/KFpP7phoZkEgM7E74tAAAAAEhExaulr/5qbU+4S7LZ23ypmWvrusb3d8owGM/emRDaAQAAACDRmKb0zvVSsFYaVCgNmdDmSwVDpuasp2t8Z0VoBwAAAIBE8/0b0uq5ksMjnfyotBfV8YVFQe30mkpzSaNz216tR3zwZxYAAAAASCTVxdIHt1jbx94k9Ry8V5ebWTdr/Ohcu5YXhyRJK0tDe3VNdBxCOwAAAAAkkhl3SNU7pF7DpSN/s9eXCy/1Nnt9ULPXV0Ud85AIEx5fEQAAAAAkirXzpe9etrZPfUJyuPbqcsuLg1pREpLNkEKmNHmUS31TrVHSHoeUl8qI6URHaAcAAACARBCold6ZYm0fconU/4i9vuSfv/NJko7Is+uzzUH1TbWpgCXfOhW+LQAAAABIBHMekHaskFJyrCXe9tL68pDeXuWXJJ09zLnX10N8ENoBAAAAIN5Wz5PmP2ltn/KYlNRjry/57MJahUxpbL5dQ3owa3xnRfd4AAAAAGgPO1dJtRX1r91p9TPBNzzmLZPe/JUkU9r3VCkj3zq+F7PGb6sK6c3lVpV98ih3m6+D+CO0AwAAAECs7VwlPX1w4/3Xfms9N3VMkn78n/UIn9vG4P63xT75QtYyb4flObRke7BN10H8EdoBAAAAINbCVfRjbrQq52UbpE8eja68H3OjVLpe+v4NybBLR18vZfSrP3fTAuv8hhX6FijxhvTKD9YEdFeP2rvZ5xF/hHYAAAAAaC8Z+VLPIU0fszmlH962tg+5RBo0ztp2JlnP/7mi/txWVN1f+N6n6oC0X0+bxuUT+To7vkEAAAAAiIcFL0jBWqnPKGm/0+r3p/eVfv6c5K9pukK/G5U+U9OWWFX2yaPcMgyjPVqODkRoBwAAAICOZIas58ptUnJP6egbJGOXhb3S+0a/3rEi+rnOmrKgqnz1r19Z5lO5TxqUadOJBY7I8ZWloRh/CHQUQjsAAAAAdKQF06xnm0MqvG33y7s11VW+bv+asqAKX6tq8m2/O9yt9RWhRsc9JMBOh68MAAAAADrKmo+lb1+0tkecLWXvs/vzG3aVD3MmSel9VVU3I/zkUS7lphh6coFPO2pMHd3XrhMGOiMzxk8e5VLfVJs8Dikv1dbUXZDACO0AAAAA0FHmPFC/3e+wlr1n167yu+ibatP3O4LaUWNKkn59kLvR8YJMwnpnxTcHAAAAAO3NW2Y9B2qkPs2s0d5Ca8qCWrI9GBmnvqMmpDeX+yPHd3rNqOPo3Ki0AwAAAEB78lVJXz1nbaf1kSbcKb102u7f04ymxrG/tdIvf0ga1sOm5SUhTZldE3WcceydG18fAAAAADS0c1X0EmvutBavkd5I0C/NuU8q32y9HjNZqixqc9PCM8WHx6l/uy2gN1cE5LZLkw92KRCSvIH68xnH3vkR2gEAAAAgbOcq6ekmuq9f+23bgvvCf0hbv5ccHinglT66rf5YeGb4NuibalOGW3p3tZXQz97XqV7JhPOuiNAOAAAAAGHhCvsxN0oZ+VLZBumTR6Mr7y1hWpPCacsia2m38bdLKdn1s8DXzQDfVqZp6vnvA6oJSEPq1mRH18Q3CwAAAAC7ysiXeg5p23tDIWn+E3UvDOsPAHkHxaplkqTvd4S0YFtQdkO68iCXbIYR0+sjcdB/AgAAAABiJRSU3r5W+mG69XrE2dLAY2J+m/+tsmaLP32oQ/npxLqujEo7AAAAAMRC0C/999fSkn9Lhk0yQ1L/Me1yqyq/1DfV0GlDnO1yfSQO/iQDAAAAAHvL75XeuMQK7DaHdNyd7XKbBdusiecMWd3inXa6xXd1VNoBAAAAYG+UbpBev1Da/J1kd0vnvCyl9o75bbwBU898Z635dkQfu/bJssf8Hkg8VNoBAAAAoK1Wz5P+MtYK7ElZ0gVvSvtMbJdbPfNdrbZVW7PSnzCQ+mt3wTcNAAAAAK1lmtJnT0sz77TGrucdJJ39stRjQLvcbl1ZSM8u8kVeu+kW38jK4oC0qUwpbocKslPi3ZyYIbQDAAAAQGsUr5He+620cqb1+qBfSqc8Zq293k7u+cwrX1AamWPTwqJQu92nM/I4rN4HUz4qkT76VJI057fjukxwJ7QDAAAAQEsE/VZ1fd5DUsAr2V3SxAek0ZdLhiHtXCXVVljn7lgRs9u+8kOtZq0PyGFIkwqcWlhUG7NrdwV5SSE95pgq7zG3aZOyNXXOSlXVBuLdrJghtAMAAADAnmxZJE2/Sir6wXo98BjplMel7KHW652rpKcPbvy+vay+/7gzoNs+sUJ6wJQe/NLa9pDkouQZxVIPh2S2X2+HeOGrBgAAAND9NKyKu9OknoObPq94tfX8v99Yz0lZVnX9oHOt6npY+FrH3Chl5FvbziQpve9eNfOlpX5JUopT+u1ot9x2Qx6HlJfKnOLdBaEdAAAAQPfSVFX82m+jg3vxamnBi9KW76zXNod08EVS4e+llJ7NXzsjX+o5JCbNXF0a1JvLrdB+6mCn9u3JEm/dEaEdAAAAQPfSsCouSZ88au0LBa0l3CTps6esZ8NmzQ5/zivSsBM7rImBkKkb53jlq5tz7sBsKuvdFaEdAAAAQPcU7sYuSV/+WVr9sVSx2Xpts0uDj5PyD5dm3yul5XZo055b5NN3RUElO6TqgGQYLPHWXfHnGgAAAADdU22F9NNH1vai1+oDuyRNfFA68jdS8m66wreTZTuDeuIba8K5Xx/k6vD7I7FQaQcAAADQvZRttJ5n3S2F6pYGS86WBo2T+o2WkjIbTyAXXsJtd5PWxYAvaOqGOTXyh6QJAxwa39+hxxf42u1+SHyEdgAAAADdw4avpPlPSj++a70OBaTsfaQDzpTyj7C6xO8qvGTbf66o37frpHUx9PS3tVq2M6QeHkMPHOtRUZXZLvdB50FoBwAAANA5NVy2bVfhingoJK34wJpYbv3n0eeMuVYaekL00m27Su8r/fw5yV8jlW2wJq3btKBFy8WtKQuqqkGRPMUlFWQ0PwP8/I0BPfOd9Yb7j/EoJ9mmoqpg821Dt0BoBwAAAND5NLVs267G3yEtfq2+a7vNKY04Rxp6vPTGxVbYbskEb+Gu8k1V3SWr8r6LNWVBFb5W1Wj/nHNTmgzun2wM6PIPqhU0pdOHOHXSIOee24VugdAOAAAAoPNpuGxbw1ng/dXS8vektZ9Ks++x9rnTpf1/Lg3/mZSSXR/iW6th1V2qr7w3Ue0PV9gnj3Kpb6pNmypDmvqdL6ryHjZvQ0BXfFgtX1A6rr9DD43ztK196JII7QAAAAA6r4x8qecQqWq79MNb0ooPpUBdqE7pJR11nVRwrPTcsdK3L0a/N1w5b41dJ6jbg76pNhVkNr9o15z1fv36oxr5gtLxAx3604Qkue0s74Z6hHYAAACgM9h1/HY7z2K+xzbE4/5NKVlrhfU1H0tm3fjvtFypYqt07j+l/NHS5oXW/oZVeWdSqwN4s9pQuf9xZ0D/XhHQC0t8CoSkMX3sunqkSz8Vh6LOW1kaauYK6C4I7QAAAECia278djvOYt6iNnTk/RuqKZGWvGltz3+ifn/ugdL+Z0ieTOnd6yX7LuPCw1X5WGlqjPseqvfVflMPflGj5xb5o/Z/vjmon0+vbvZ9HpJbt8VXDwAAACS6Xcdv72YsdYe0Qer4+3vLra7vy96SfpohBbzWfptTKjhGGnay1GuYtW/nyo5p065j3Jup3lf4TC3dYfUCuOzDalXUjWtPc0mF+Q6NzrPLvpsJ8TwOKS+1+S726NoI7QAAAEBnEetKcVvb0BJ725XeNKXty6V1n0orPpJWz5GCDWZxyxosFa+Sjr9byh3RumvHUhMh3TRN/VRihfQnF9RqW3X9WusVPqlPqqHNlab+b7RbQ7KaXwIOkAjtAAAAAGKttV3pQyGpbL204ycrqG/8Slr3mTW5XEPZ+1gzwO/3M+s9fx0nOZPb5SO0xfrykKb/5Nf0lX6trhuLHg7svZIMba8xddsRbh2WZ9dp/62W3caEc9gzQjsAAACAvdewsh6emG3XrvS+aqvr+o4VVkDfsaJ+O1jb+JqOJCn3ACn3IKsLfO6B9cE/PLlcnK0pC2pjeUgvLfVr5rqAwjV1p03yh6Tz9nVqbH+HimtM3fqJV2P6EMHaTdkGKRSIdytijt8YAAAAAHunuYnyKrZIZRut7VfPkSq3tux6R1wt7XeaNaHcM4dLG7+WvvmbdSxek981YXlxQBPfaHryOH/dpO+j8+zKcBsqrjGbPA8x4HBbzx8/Ipm5kq6wfu/6ZsS1WbFCaAcAAAC6s71ZSs5fLW34Slr+vvU6Lc+a2T08SdzCV+vPDQd2T4aU2b/uMUAy7NLnT0tHXy8ZNqsqP+Icqc/I6KXaJOvYpgVWe1uyzFr4nDYsydacNWVBVdaa+nxzUM8tqh9j3y/N0CmDnOqfXj9hHBPIdZCUbOt3JFArbS+TlknyVcW7VTFDaAcAAAC6q5YuJWea1rrnkjWDe/lma/uFkyQ1qCBXbLGeDZuU2lvK6GdVy1fOqD/HWyZt/d56NNRr3/pZ2HeVkd/08mpS00ustebcVlhTFlTha43D4HnDnTplsEO23cwAj3aWkm09V3S9if0I7QAAAECi2Juqd1s0t5TcuvnSqtlS0Q/StqVS0TKpttw6d8X7DS5gSqm5Uo/+VsV91IVS/uFSeh/J7qo/7cCzmg/kUv1SaeGl2pqqkO+6vFrD9+2qBeeuKQuqqq5QnuKSCjJ2H/a2V4f00Jf14+4dhnR0P7smDnRoYGbLg+LKugnqgJYitAMAAACJYHdV71jx10g1pZK31HpeN9/av+5zKTBHKlljvX772sbvtTmsSb76HmqF32VvSRe+JQ0eZ3Vj/8tYqe8hUo+Bjd/bVLBuyp4q5C29zh7ObapiPufclEbBPRgy9cnGoP71o08z1wUi49QPyLbpyoNc6pXc8q7vnrrkNWV2TaN9wO7wawIAANBd7O262XXWrFymqkrrOimpaSoYMjxWLYyvjq5yh5mm1WV8y0Lr9ZAJ1jjviq2q2vKj9OrFsvvKZTMdMj68XTYzKCPok2FKxl/HW++JdMtu0D3bsEUfCwWlkL/pNqz/LPp1Upa1vFrOcGnAUVLv/a0xwn+fII26wDpn2VtSUuZefvhdtKaaruhqeVP8QVPVAckbMOWwSwPT7bLbpHVlVvr+1YFO2ST97Xu/dlabMs2gNlSY+m5bQGvKQvpsU1DbG0wgNyzLpuXFIf1yeOsCu2SNbX+s0CNv3eTmjHdHSxHaAQAAuoPWrpvdjDUrl6nwb6sb7NmuOZer8wf3lo7tbg3TtCZkqymRyrdIFZubeK7b9jeo+q6cGdlMMSTtrBv7bUjyN/ijgiHJDNbdq21NjBh2svU57U6re3xNsbThC+tx5LXWsY5aYq2F1fTmxpe3xvPf1/8R46y3m54FPuyvEz3KS7HrlP+0/Z6EdLRFlwntU6dO1cMPP6ytW7fqoIMO0tNPP63DDjss3s0CAABIDA3HLkv162a3UrjCPnmgNRHZ1LV9IvsSkmlKQZ9VuQ14m38u+tE6f/BxkitVqt5hdR2f+wermhz0W4+Q37pe+HXQZ3UZb7gvUGNVzr1l1v6WcmdItWVS7wMUTO+vNVVO/WltHw3tmyN7UrqSV72r9CFHyZ7eW5tqHHp+SVAvHV2ifTN2WZe6YpP01V+lX74u5eyn+kRvSKXrpGkn149hlxpXsrP3sX4u4fHtbfg9aQ9VflM/7gxq6Y6QftgZ1Ddbgy1+r92QgqaU5JBCphQMSYEm/tCR7JB6JRtaV25qRC+bhmfZlemRnlvkV15K15vgDJ1Dlwjt//rXv3TDDTfo2Wef1eGHH64nnnhCEydO1PLly5WTkxPv5sVWvLptIRrfAxB7u/nnas2OKlXV1v9PaYrboYLslI5uIVqjM/97cg9tb/j7mJC/i+Hqbm2F5C23Jg+rLZe2LLaOF6+W/F6VmskK/e922exOyQzKbrPLnpSmyoBd1UGbKr0+VQVsqgratcNMV5GytC2Yqk3VDg0xXJq50alq06V0VenCN7xK+e86pdiDSnWE1MMtZblDynKHlO2RstymeroDsgdqlG7UqIe9Vm7DL5evVJl2n1UtDoUkM2B14TaDdV2567bNUPRrGVZFOByWayulYG19aA6/P+iTGfDKMFsx8deqWdGvv399778Twy4lZ0kpvaTkbGuW65ReMpOzVZo8QOuS99fKqiStWLdRKxZ+qp+2H6jN6+z1xfMN4Y3LpeVSilNKdRoqkqkXd+TomCSH9ulh04B0m5x2Q9qZaZ2e2lvKrAvm4d/r8KzvGflSzyFNt7c148YbTBi3JpSrqhKH9bNXyyZ3C1teHNC2SlNVfikgU3bD0KaKkDZVmtpYEdLqspDWlIaa7FDgtksFGTb1STXUK9mm3smGclIMZbgMeRyGPA5pQ7mpWz/x6sFjkzQk06aVpSFNmV2j+452K2RKd8yv1eunJmt0nl1Ld4R0yn+qdO6+LhVk2rSmNCTJzwRyiJsuEdofe+wxXXHFFbr00kslSc8++6zeffddPf/88/rd734X59bFRk3Zds197XFp41eNDxaMldypkmzW2CXDJtMwrNc2myTDGstUd6z+tSHJkGk0cU7dsfrz6pmmoeb7YEXvN83dnNrEe81G5+7+zdHnN3Vu3T4zVLdZ1yDTrG9cM8+mada9JxR9DV9lo/+gm5I08FjJmVx/z8jP0dbgZ2u9Nnf9PiLfU4Nz616bMiRb3XVaoDW941rbk67x97OLhr8rrbx4fNrd+EDbfgdb0CJz17OaeI+5xzNa8CW07Byz0TnNv8ds7pymG1j3z07Df6ZCqv/nLrTLP2shBUJStbdWNZuWqkZu1citatOtGrlUk9pfVUGndlT5ZcpQSIZMGbLJVF66Q6lOyWMz5bGH5LGb8tglj92U227WPatun6xleAxDRt3D+udNkmx1r8PHbJFtaZfzm/0ZmtE/ErPJvdarXf6fL/pfY2aLft5mo2Om9e+KhnfdtTFR+5q+TvTxkBQyZTb1/YX3hazv0Iz6bkMya6uk9Z83uE9d2/ocLLmSJNmsOzf4748po9F/e0w1+PehJFN131nD/27t8hy5V12bzKh/51ttjPxzW7cvFDLlDRqqCRqqqfWrumiVvKZb1ar7PTTd8iX3lmlzKBAyVVxlBXa7gnIYIQ3IdCnVpcjvn8dmyuMwI797brs1dtVha/D7VPe7Jdsuv2914dQwQzLMgGQGZYSs0GqYAZlBv8xArUx/rfUceVjhNOT3qTYQlDdkl1dOeU23vHLVPZz6/+3de1BU5f8H8PdhYW+wy81dEAYloEySHxYMhrdkcthKHf2ljVNm2IWyyDI18TbezUZnSmOytBql5jellVONMRIRZpOkjUKlBn77wkIi4JVLXPb6/P5AVzcurQHuLrxfM2fiPOd5zn7O9nH3fPY5e7ZdZKG1RIEWqNAixuIvowqt4uo6lLC7+F4DALhxctcGXHF90hMAoEY7NGiFRmpFANqgkdqgvbquQSsCpDbIYYUfbPCDFb6wQkAOO3xgu7pY4Iu2a68XUKBNyNGO6//v2oUc4mq+CUgQAHxhhwrtUElmKGCBEmYoJQsUMHf8rQ2F0s8XSpihuFwGhWS5lrFX93A1fxyZeWN7x98CEqyQwSpksEIGC3zR3KhGU4MaTfBHg/DHWaFDlQhDE/wBVN3wzIwG/naVthYtUMKERgTABDlaLB2zzgDwcZkFH5d1XOLtIwF6tYQIRTAizAugP9yAgJByqKyN8D+yBUqY4SPZIWECcFYD6ZL5+jFIEiQAViFgsgImm4C5WQ2T9X9hOtIIk/I0TI0NMFky0fqTFm12X7SaV6Lt43MQqIMJfigTbwCFAHD98vGRoT5QyCSIq69vQnTMcJusQJtVoM3acSyWXtTDmyYoEKnp+cMBpW/H83XjTeAAQO0nOb5jrva79m/x72PRaSxvIEe3kiQ6n7l5FbPZDLVajc8++wwzZsxwtGdkZKChoQFffvllpzEmkwkm0/Wfa2hsbMSwYcPw559/QqvV3oqwb9q56gqkf/Afd4dBREREg4BCskAl2qGQ2aGUrNDYGhGslBDkZ4eftRnn23wwyr8BOt824K86SKpgmGVqNFl8UNPmhxCVDBbJD402BZqFEg02JS7b1bhiU6EVStjAy4xvpPG1IlQJ6GV/Qdd8Gvqh0QgN0qC2VYb/q1QjI/IchqtMED5+MPlq0Wz1wX+affFtrRJxGivarBLOm3xgsXv7b4QL+EmARUjQKW0IU9oRKLcjSG5HsMIOk1XCp1VqpA9tx1B1x6dEKpmAXulaxX++3QdttuvP0bWxVS0y5P7XH0/EtiJGY0VFsy8+/K8aGbEtGO5v6zT2Zh6Tbr3ahha8Vz0U+2aGIv5/kt0dTo+ampoQFRWFhoYGBAYGdtvP6z8junjxImw2G8LCwpzaw8LCUFZW1uWYzZs3Y926dZ3ao6Ki+iVGIiIiIqJ/a2MP2/7sYZs36+m4Puinx9z0t/WennfyfKnb3B2B65qbmwd20f5vLF++HIsWLXKs2+12XL58GaGhoV1eEkODw7VPujz5igvybswxuhWYZ9TfmGPU35hj1N88JceEEGhubkZERESP/by+aB8yZAhkMhnq6+ud2uvr6xEeHt7lGIVCAYVC4dQWFBTUXyGSl9FqtXyDoH7FHKNbgXlG/Y05Rv2NOUb9zRNyrKcZ9mu8/ocC5XI5kpKSUFh4/cZgdrsdhYWFSE1NdWNkRERERERERL3j9TPtALBo0SJkZGQgOTkZKSkp2LZtG1paWhx3kyciIiIiIiLyRgOiaJ89ezYuXLiA1atXo66uDqNHj8bBgwc73ZyOqCcKhQJr1qzp9NUJor7CHKNbgXlG/Y05Rv2NOUb9zdtyzOt/8o2IiIiIiIhooPL677QTERERERERDVQs2omIiIiIiIg8FIt2IiIiIiIiIg/Fop2IiIiIiIjIQ7Fop0Fp06ZNGDt2LNRqNYKCgrrsU11djSlTpkCtVkOv1+PVV1+F1Wp16nPo0CHcc889UCgUiIuLw549e/o/ePJKZ86cwfTp0zFkyBBotVqMHz8eRUVFTn1cyTminnz99dcYM2YMVCoVgoODMWPGDKftzDHqKyaTCaNHj4YkSSgtLXXa9uuvv2LChAlQKpWIiorCli1b3BMkeR2j0Yinn34at912G1QqFWJjY7FmzRqYzWanfswx6q23334b0dHRUCqVGDNmDI4dO+bukHrEop0GJbPZjEceeQTPP/98l9ttNhumTJkCs9mMI0eOIDc3F3v27MHq1asdfSorKzFlyhSkpaWhtLQUCxcuxDPPPIP8/PxbdRjkRaZOnQqr1YrvvvsOx48fR2JiIqZOnYq6ujoAruUcUU8+//xzzJ07F08++SR++eUX/Pjjj3jssccc25lj1JeWLl2KiIiITu1NTU1IT0/H8OHDcfz4cWzduhVr167Frl273BAleZuysjLY7Xbs3LkTp06dwptvvol3330XK1ascPRhjlFv7d27F4sWLcKaNWtw4sQJJCYmwmAw4Pz58+4OrXuCaBDbvXu3CAwM7NSel5cnfHx8RF1dnaPtnXfeEVqtVphMJiGEEEuXLhV33XWX07jZs2cLg8HQrzGT97lw4YIAIA4fPuxoa2pqEgBEQUGBEMK1nCPqjsViEZGRkeL999/vtg9zjPpKXl6euPPOO8WpU6cEAFFSUuLYtmPHDhEcHOyUU9nZ2WLEiBFuiJQGgi1btojbbrvNsc4co95KSUkRWVlZjnWbzSYiIiLE5s2b3RhVzzjTTtSF4uJiJCQkICwszNFmMBjQ1NSEU6dOOfpMnjzZaZzBYEBxcfEtjZU8X2hoKEaMGIEPP/wQLS0tsFqt2LlzJ/R6PZKSkgC4lnNE3Tlx4gRqamrg4+ODu+++G0OHDsWDDz6IkydPOvowx6gv1NfXIzMzEx999BHUanWn7cXFxZg4cSLkcrmjzWAwoLy8HFeuXLmVodIA0djYiJCQEMc6c4x6w2w24/jx407n8D4+Ppg8ebJHn8OzaCfqQl1dndOJLQDH+rXLmbvr09TUhLa2tlsTKHkFSZLw7bffoqSkBBqNBkqlEm+88QYOHjyI4OBgAK7lHFF3KioqAABr167FqlWrcODAAQQHB2PSpEm4fPkyAOYY9Z4QAvPmzcP8+fORnJzcZR/mGfWlP/74Azk5OXjuueccbcwx6o2LFy/CZrN1mUOenD8s2mnAWLZsGSRJ6nEpKytzd5g0gLiac0IIZGVlQa/X44cffsCxY8cwY8YMTJs2DbW1te4+DPJgruaY3W4HAKxcuRIzZ85EUlISdu/eDUmS8Omnn7r5KMjTuZpnOTk5aG5uxvLly90dMnmZf3OOVlNTgwceeACPPPIIMjMz3RQ5kWfwdXcARH1l8eLFmDdvXo99YmJiXNpXeHh4p7tI1tfXO7Zd+++1thv7aLVaqFQqF6Mmb+Zqzn333Xc4cOAArly5Aq1WCwDYsWMHCgoKkJubi2XLlrmUczT4uJpj1z78iY+Pd7QrFArExMSguroagGuvazQ43cxrWXFxMRQKhdO25ORkzJkzB7m5ud2+NwLMs8HsZs/Rzp07h7S0NIwdO7bTDeaYY9QbQ4YMgUwm6zKHPDl/WLTTgKHT6aDT6fpkX6mpqdi0aRPOnz8PvV4PACgoKIBWq3WcFKempiIvL89pXEFBAVJTU/skBvJ8ruZca2srgI7vTN3Ix8fHMUPqSs7R4ONqjiUlJUGhUKC8vBzjx48HAFgsFhiNRgwfPhwAc4y652qevfXWW9i4caNj/dy5czAYDNi7dy/GjBkDoCPPVq5cCYvFAj8/PwAdeTZixAjH14Fo8LmZc7SamhqkpaU5rhj6+3snc4x6Qy6XIykpCYWFhY6fRbXb7SgsLMSLL77o3uB64u474RG5Q1VVlSgpKRHr1q0TAQEBoqSkRJSUlIjm5mYhhBBWq1WMGjVKpKeni9LSUnHw4EGh0+nE8uXLHfuoqKgQarVavPrqq+L3338Xb7/9tpDJZOLgwYPuOizyUBcuXBChoaHi4YcfFqWlpaK8vFwsWbJE+Pn5idLSUiGEazlH1JOXX35ZREZGivz8fFFWViaefvppodfrxeXLl4UQzDHqe5WVlZ3uHt/Q0CDCwsLE3LlzxcmTJ8Unn3wi1Gq12Llzp/sCJa9x9uxZERcXJ+6//35x9uxZUVtb61iuYY5Rb33yySdCoVCIPXv2iNOnT4tnn31WBAUFOf26iqdh0U6DUkZGhgDQaSkqKnL0MRqN4sEHHxQqlUoMGTJELF68WFgsFqf9FBUVidGjRwu5XC5iYmLE7t27b+2BkNf4+eefRXp6uggJCREajUbce++9Ii8vz6mPKzlH1B2z2SwWL14s9Hq90Gg0YvLkyeLkyZNOfZhj1Je6KtqFEOKXX34R48ePFwqFQkRGRorXX3/dPQGS19m9e3eX52d/n2dkjlFv5eTkiGHDhgm5XC5SUlLETz/95O6QeiQJIYRbpviJiIiIiIiIqEe8ezwRERERERGRh2LRTkREREREROShWLQTEREREREReSgW7UREREREREQeikU7ERERERERkYdi0U5ERERERETkoVi0ExEREREREXkoFu1EREREREREHopFOxEREfWLQ4cOQZIkNDQ0/Ot9TJo0CQsXLuyzmIiIiLwNi3YiIiIvJElSj8vatWs7jWltbcXy5csRGxsLpVIJnU6H++67D19++aWjT3R0NLZt23bT8XRVXI8dOxa1tbUIDAz8x/HdFfj79+/Hhg0bbjoeIiKigcLX3QEQERHRzautrXX8vXfvXqxevRrl5eWOtoCAgE5j5s+fj6NHjyInJwfx8fG4dOkSjhw5gkuXLvVLjHK5HOHh4b3aR0hISB9FQ0RE5J04005EROSFwsPDHUtgYCAkSXJq66po/+qrr7BixQo89NBDiI6ORlJSEhYsWICnnnoKQMdseVVVFV555RXHjD0AXLp0CY8++igiIyOhVquRkJCAjz/+2LHfefPm4fvvv8f27dsd44xGY6fZ86qqKkybNg3BwcHw9/fHXXfdhby8PBiNRqSlpQEAgoODIUkS5s2b54jpxhl8k8mE7OxsREVFQaFQIC4uDh988EE/PMNERESegTPtREREg0R4eDjy8vLw8MMPQ6PRdNq+f/9+JCYm4tlnn0VmZqajvb29HUlJScjOzoZWq8XXX3+NuXPnIjY2FikpKdi+fTvOnDmDUaNGYf369QAAnU4Ho9HotP+srCyYzWYcPnwY/v7+OH36NAICAhAVFYXPP/8cM2fORHl5ObRaLVQqVZfH8MQTT6C4uBhvvfUWEhMTUVlZiYsXL/bdk0RERORhWLQTERENErt27cKcOXMQGhqKxMREjB8/HrNmzcK4ceMAdFyKLpPJoNFonC5rj4yMxJIlSxzrCxYsQH5+Pvbt24eUlBQEBgZCLpdDrVb3eDl8dXU1Zs6ciYSEBABATEyMY9u1y+D1ej2CgoK6HH/mzBns27cPBQUFmDx5cqd9EBERDUS8PJ6IiGiAqa6uRkBAgGN57bXXAAATJ05ERUUFCgsLMWvWLJw6dQoTJkz4xxu92Ww2bNiwAQkJCQgJCUFAQADy8/NRXV19U3G99NJL2LhxI8aNG4c1a9bg119/vanxpaWlkMlkuO+++25qHBERkTdj0U5ERDTAREREoLS01LHMnz/fsc3Pzw8TJkxAdnY2vvnmG6xfvx4bNmyA2Wzudn9bt27F9u3bkZ2djaKiIpSWlsJgMPQ4pivPPPMMKioqMHfuXPz2229ITk5GTk6Oy+O7u2SeiIhoIGPRTkRENMD4+voiLi7OsfR0B/b4+HhYrVa0t7cD6Ljju81mc+rz448/Yvr06Xj88ceRmJiImJgYnDlzxqlPV+O6EhUVhfnz52P//v1YvHgx3nvvPcd4AD3uIyEhAXa7Hd9///0/Pg4REdFAwaKdiIhokJg0aRJ27tyJ48ePw2g0Ii8vDytWrEBaWhq0Wi2Ajt9pP3z4MGpqahw3eLv99ttRUFCAI0eO4Pfff8dzzz2H+vp6p31HR0fj6NGjMBqNuHjxIux2e6fHX7hwIfLz81FZWYkTJ06gqKgII0eOBAAMHz4ckiThwIEDuHDhAv76669O46Ojo5GRkYGnnnoKX3zxBSorK3Ho0CHs27evr58qIiIij8GinYiIaJAwGAzIzc1Feno6Ro4ciQULFsBgMDgVvevXr4fRaERsbCx0Oh0AYNWqVbjnnntgMBgwadIkhIeHY8aMGU77XrJkCWQyGeLj46HT6br8vrvNZkNWVhZGjhyJBx54AHfccQd27NgBoONmd+vWrcOyZcsQFhaGF198sctjeOeddzBr1iy88MILuPPOO5GZmYmWlpY+eoaIiIg8jySEEO4OgoiIiIiIiIg640w7ERERERERkYdi0U5ERERERETkoVi0ExEREREREXkoFu1EREREREREHopFOxEREREREZGHYtFORERERERE5KFYtBMRERERERF5KBbtRERERERERB6KRTsRERERERGRh2LRTkREREREROShWLQTEREREREReaj/B/uvx2C1+SqmAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA+0AAAK9CAYAAABRvo1QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAACM8klEQVR4nOzdd3xUVf7/8fdMeu+dkITeO0IUpQpS3GXBVVwU9eeqXwRcbKvYxYIVbCh2FHUVu6IgHZUmRTqElhAgJAHSe5n5/REYjRBIwiRzk7yej8c8JnPvued+bhJ2fefcc67JarVaBQAAAAAADMfs6AIAAAAAAMDZEdoBAAAAADAoQjsAAAAAAAZFaAcAAAAAwKAI7QAAAAAAGBShHQAAAAAAgyK0AwAAAABgUIR2AAAAAAAMitAOAAAAAIBBEdoBADjlsccek8lkqpdzDRgwQAMGDLB9XrlypUwmk7744ot6Of+NN96o2NjYejlXbeXl5enf//63wsPDZTKZNHXqVEeXVC0mk0mPPfaYo8sAADQShHYAQKM0d+5cmUwm28vd3V2RkZEaNmyYXnnlFeXm5trlPCkpKXrssce0ZcsWu/RnT0aurTqefvppzZ07VxMnTtS8efN0/fXXV9k2Nja20s87NDRUl156qb7++usqj7njjjtkMpm0f//+Kts8+OCDMplM2rZt2wVdCwAAtUVoBwA0atOnT9e8efP0xhtvaMqUKZKkqVOnqnPnzmcEsYceekiFhYU16j8lJUWPP/54jYPx4sWLtXjx4hodU1Pnqu3tt99WQkJCnZ7/Qi1fvlx9+/bVo48+quuuu049e/Y8Z/tu3bpp3rx5mjdvnu655x6lpKRozJgxmjNnzlnbjx8/XpL0ySefVNnn//73P3Xu3FldunSp/YUAAHABCO0AgEZt+PDhuu6663TTTTdp2rRp+umnn7R06VKlp6frb3/7W6WQ7uzsLHd39zqtp6CgQJLk6uoqV1fXOj3Xubi4uMjNzc1h56+O9PR0+fv7V7t9VFSUrrvuOl133XX673//q9WrV8vLy0uzZs06a/s+ffqoVatW+t///nfW/WvXrlViYqIt3AMA4AiEdgBAkzNo0CA9/PDDOnTokD766CPb9rPNaV+yZIn69esnf39/eXt7q23btnrggQckVcxD7927tyTppptust2aPXfuXEkV89Y7deqkTZs26bLLLpOnp6ft2L/OaT+tvLxcDzzwgMLDw+Xl5aW//e1vOnz4cKU2sbGxuvHGG8849s99nq+2s81pz8/P1913363o6Gi5ubmpbdu2euGFF2S1Wiu1M5lMmjx5sr755ht16tRJbm5u6tixoxYtWnT2b/hfpKen6+abb1ZYWJjc3d3VtWtXffDBB7b9p+f3JyYm6ocffrDVnpSUVK3+TwsPD1f79u2VmJhYZZvx48drz5492rx58xn7PvnkE5lMJl177bUqKSnRI488op49e8rPz09eXl669NJLtWLFivPWUdX6AVWtofDRRx+pZ8+e8vDwUGBgoMaNG3fG7wAAoOkgtAMAmqTT86PPdYv6zp07NWrUKBUXF2v69Ol68cUX9be//U2rV6+WJLVv317Tp0+XJN166622W7Mvu+wyWx8nT57U8OHD1a1bN7300ksaOHDgOet66qmn9MMPP+i+++7THXfcoSVLlmjIkCE1vm2/OrX9mdVq1d/+9jfNmjVLV1xxhWbOnKm2bdvq3nvv1V133XVG+19//VW33367xo0bp+eee05FRUUaO3asTp48ec66CgsLNWDAAM2bN0/jx4/X888/Lz8/P9144416+eWXbbXPmzdPwcHBlW55DwkJqdH3oLS0VIcPH1ZQUFCVbaq6Rb68vFzz58/XpZdequbNmysnJ0fvvPOOBgwYoGeffVaPPfaYjh8/rmHDhtl1zYCnnnpKEyZMUOvWrTVz5kxNnTpVy5Yt02WXXaasrCy7nQcA0IBYAQBohN5//32rJOuGDRuqbOPn52ft3r277fOjjz5q/fP/Nc6aNcsqyXr8+PEq+9iwYYNVkvX9998/Y1///v2tkqxz5sw5677+/fvbPq9YscIqyRoVFWXNycmxbZ8/f75VkvXll1+2bYuJibHecMMN5+3zXLXdcMMN1piYGNvnb775xirJ+uSTT1Zqd9VVV1lNJpN1//79tm2SrK6urpW2bd261SrJ+uqrr55xrj976aWXrJKsH330kW1bSUmJNT4+3urt7V3p2mNiYqwjR448Z39/bjt06FDr8ePHrcePH7du3brVOm7cOKsk65QpU855bO/eva3NmjWzlpeX27YtWrTIKsn65ptvWq1Wq7WsrMxaXFxc6bjMzExrWFiY9f/9v/9Xabsk66OPPmr7/Nfv9Wl//X1LSkqyOjk5WZ966qlK7bZv3251dnY+YzsAoGlgpB0A0GR5e3ufcxX50/Opv/32W1ksllqdw83NTTfddFO120+YMEE+Pj62z1dddZUiIiL0448/1ur81fXjjz/KyclJd9xxR6Xtd999t6xWqxYuXFhp+5AhQ9SyZUvb5y5dusjX11cHDx4873nCw8N17bXX2ra5uLjojjvuUF5enlatWlXra1i8eLFCQkIUEhKirl276vPPP9f111+vZ5999pzHXXfddTpy5Ih+/vln27ZPPvlErq6u+uc//ylJcnJysq1BYLFYlJGRobKyMvXq1eust9bXxldffSWLxaKrr75aJ06csL3Cw8PVunXrat2KDwBofAjtAIAmKy8vr1JA/qtrrrlGl1xyif79738rLCxM48aN0/z582sU4KOiomq04Fzr1q0rfTaZTGrVqlWN53PX1KFDhxQZGXnG96N9+/a2/X/WvHnzM/oICAhQZmbmec/TunVrmc2V/xOkqvPURJ8+fbRkyRItXbpUa9as0YkTJ/Thhx/Kw8NDkpSamlrpdXrKwbhx4+Tk5GS7Rb6oqEhff/21hg8froCAAFv/H3zwgbp06SJ3d3cFBQUpJCREP/zwg7Kzs2td85/t27dPVqtVrVu3tv3x4fRr9+7dSk9Pt8t5AAANi7OjCwAAwBGOHDmi7OxstWrVqso2Hh4e+vnnn7VixQr98MMPWrRokT777DMNGjRIixcvlpOT03nPczow2tPZFi+TKuZhV6cme6jqPNa/LFpXn4KDgzVkyJAq90dERFT6/P777+vGG29UaGioLr/8cn355ZeaPXu2vv/+e+Xm5lZaNf6jjz7SjTfeqNGjR+vee+9VaGionJycNGPGDB04cOCcdZ3r5/VnFotFJpNJCxcuPOv319vb+5znAQA0ToR2AECTNG/ePEnSsGHDztnObDZr8ODBGjx4sGbOnKmnn35aDz74oFasWKEhQ4ZUGchqa9++fZU+W61W7d+/v9JzwgMCAs66KNmhQ4fUokUL2+ea1BYTE6OlS5cqNze30mj7nj17bPvtISYmRtu2bZPFYqk02m7v85zNkiVLKn3u2LGj7evx48dr0aJFWrhwoT755BP5+vrqyiuvtO3/4osv1KJFC3311VeVvq+PPvroec97rp/Xn7Vs2VJWq1VxcXFq06ZNdS8LANDIcXs8AKDJWb58uZ544gnFxcWd8xncGRkZZ2zr1q2bJKm4uFiS5OXlJUl2W9n7ww8/rDTP/osvvtCxY8c0fPhw27aWLVtq3bp1KikpsW1bsGDBGY8Fq0ltI0aMUHl5uV577bVK22fNmiWTyVTp/BdixIgRSk1N1WeffWbbVlZWpldffVXe3t7q37+/Xc5zNkOGDKn0+vPI++jRo+Xp6anXX39dCxcu1JgxY+Tu7m7bf3rk+893Eqxfv15r164973lbtmyp7Oxsbdu2zbbt2LFj+vrrryu1GzNmjJycnPT444+fcceC1Wo978r8AIDGiZF2AECjtnDhQu3Zs0dlZWVKS0vT8uXLtWTJEsXExOi7776rFMz+avr06fr55581cuRIxcTEKD09Xa+//rqaNWumfv36SaoIZP7+/pozZ458fHzk5eWlPn36KC4urlb1BgYGql+/frrpppuUlpaml156Sa1atdItt9xia/Pvf/9bX3zxha644gpdffXVOnDggD766KNKC8PVtLYrr7xSAwcO1IMPPqikpCR17dpVixcv1rfffqupU6ee0Xdt3XrrrXrzzTd14403atOmTYqNjdUXX3yh1atX66WXXjrnGgN1ydvbW6NHj7bNa//rH3NGjRqlr776Sv/4xz80cuRIJSYmas6cOerQoYPy8vLO2fe4ceN033336R//+IfuuOMOFRQU6I033lCbNm0qLWLXsmVLPfnkk5o2bZqSkpI0evRo+fj4KDExUV9//bVuvfVW3XPPPfa/eACAsTlu4XoAAOrO6Ue+nX65urpaw8PDrZdffrn15ZdfrvRosdP++giuZcuWWf/+979bIyMjra6urtbIyEjrtddea927d2+l47799ltrhw4drM7OzpUesda/f39rx44dz1pfVY98+9///medNm2aNTQ01Orh4WEdOXKk9dChQ2cc/+KLL1qjoqKsbm5u1ksuucS6cePGM/o8V21newxZbm6u9c4777RGRkZaXVxcrK1bt7Y+//zzVovFUqmdJOukSZPOqKmqR9H9VVpamvWmm26yBgcHW11dXa2dO3c+62PpavrIt+q2rcoPP/xglWSNiIio9Pg3q9VqtVgs1qefftoaExNjdXNzs3bv3t26YMGCs34f9ZdHvlmtVuvixYutnTp1srq6ulrbtm1r/eijj874fTvtyy+/tPbr18/q5eVl9fLysrZr1846adIka0JCwgVdHwCgYTJZrQ5cMQYAAAAAAFSJOe0AAAAAABgUoR0AAAAAAIMitAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQRHaAQAAAAAwKGdHF2AEFotFKSkp8vHxkclkcnQ5AAAAAIBGzmq1Kjc3V5GRkTKbqx5PJ7RLSklJUXR0tKPLAAAAAAA0MYcPH1azZs2q3E9ol+Tj4yOp4pvl6+vr4GoAAAAAAI1dTk6OoqOjbXm0KoR2yXZLvK+vL6EdAAAAAFBvzjdFm4XoAAAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgmNMOAAAAAOdhtVpVVlam8vJyR5eCBsLJyUnOzs4X/FhxQjsAAAAAnENJSYmOHTumgoICR5eCBsbT01MRERFydXWtdR+EdgAAAACogsViUWJiopycnBQZGSlXV9cLHjlF42e1WlVSUqLjx48rMTFRrVu3ltlcu9nphHYAAAAAqEJJSYksFouio6Pl6enp6HLQgHh4eMjFxUWHDh1SSUmJ3N3da9UPC9EBAAAAwHnUdpQUTZs9fm/4zQMAAAAAwKAI7QAAAAAAGBShHQAAAABQI3PnzpW/v/8F92MymfTNN99ccD+NGaEdAAAAAJqgG2+8UaNHj3Z0GTgPQjsAAAAAAAZFaAcAAAAAVDJz5kx17txZXl5eio6O1u233668vLwz2n3zzTdq3bq13N3dNWzYMB0+fLjS/m+//VY9evSQu7u7WrRooccff1xlZWX1dRmNAqEdAAAAAFCJ2WzWK6+8op07d+qDDz7Q8uXL9d///rdSm4KCAj311FP68MMPtXr1amVlZWncuHG2/b/88osmTJig//znP9q1a5fefPNNzZ07V0899VR9X06DRmgHAAAAAFQydepUDRw4ULGxsRo0aJCefPJJzZ8/v1Kb0tJSvfbaa4qPj1fPnj31wQcfaM2aNfrtt98kSY8//rjuv/9+3XDDDWrRooUuv/xyPfHEE3rzzTcdcUkNlrOjCwAAAAAAGMvSpUs1Y8YM7dmzRzk5OSorK1NRUZEKCgrk6ekpSXJ2dlbv3r1tx7Rr107+/v7avXu3LrroIm3dulWrV6+uNLJeXl5+Rj84N0I7AAAAAMAmKSlJo0aN0sSJE/XUU08pMDBQv/76q26++WaVlJRUO2zn5eXp8ccf15gxY87Y5+7ubu+yGy1COwAAAADAZtOmTbJYLHrxxRdlNlfMqP7rrfGSVFZWpo0bN+qiiy6SJCUkJCgrK0vt27eXJPXo0UMJCQlq1apV/RXfCBHaAQAAAKCJys7O1pYtWyptCw4OVmlpqV599VVdeeWVWr16tebMmXPGsS4uLpoyZYpeeeUVOTs7a/Lkyerbt68txD/yyCMaNWqUmjdvrquuukpms1lbt27Vjh079OSTT9bH5TUKLEQHAAAAAE3UypUr1b1790qvefPmaebMmXr22WfVqVMnffzxx5oxY8YZx3p6euq+++7Tv/71L11yySXy9vbWZ599Zts/bNgwLViwQIsXL1bv3r3Vt29fzZo1SzExMfV5iQ2eyWq1Wh1dhKPl5OTIz89P2dnZ8vX1dXQ5AAAAAAyiqKhIiYmJiouLYx42auxcvz/VzaGMtAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQbF6fENz8oBUnFu9tm4+UlDLuq0HAAAAAFBnCO0NyckD0qs9anbMlM0EdwAAAABooAjtDcnpEfZL75b8os/dNvuw9MuL1R+VBwAAAAAYDqG9IfKLloJaOboKAAAAAEAdYyE6AAAAAAAMipF2AAAAAKiho1mFyswvqbfzBXi5Ksrfo97OB+MgtAMAAABADRzNKtTgF1eqqNRSb+d0dzFr2d0D6i24JyUlKS4uTr///ru6detWrWPmzp2rqVOnKisry6F1NDaEdgAAAACogcz8EhWVWjRpYKt6CdFHswo1e8V+ZeaX1Ph8hw8f1qOPPqpFixbpxIkTioiI0OjRo/XII48oKCioyuOio6N17NgxBQcHV/tc11xzjUaMGFGj+nB+hHYAAAAAqIUofw/FBXs5uowqHTx4UPHx8WrTpo3+97//KS4uTjt37tS9996rhQsXat26dQoMDDzjuJKSErm6uio8PLxG5/Pw8JCHB7fw2xsL0QEAAABAIzRp0iS5urpq8eLF6t+/v5o3b67hw4dr6dKlOnr0qB588EFJUmxsrJ544glNmDBBvr6+uvXWW5WUlCSTyaQtW7bY+vvuu+/UunVrubu7a+DAgfrggw9kMplst8PPnTtX/v7+tvaPPfaYunXrpnnz5ik2NlZ+fn4aN26ccnP/eCz1okWL1K9fP/n7+ysoKEijRo3SgQMH6uPb02AQ2gEAAACgkcnIyNBPP/2k22+//YzR7/DwcI0fP16fffaZrFarJOmFF15Q165d9fvvv+vhhx8+o7/ExERdddVVGj16tLZu3arbbrvNFvrP5cCBA/rmm2+0YMECLViwQKtWrdIzzzxj25+fn6+77rpLGzdu1LJly2Q2m/WPf/xDFkv9rRdgdNweDwAAAACNzL59+2S1WtW+ffuz7m/fvr0yMzN1/PhxSdKgQYN099132/YnJSVVav/mm2+qbdu2ev755yVJbdu21Y4dO/TUU0+dsw6LxaK5c+fKx8dHknT99ddr2bJltuPGjh1bqf17772nkJAQ7dq1S506dar+BTdijLQDAAAAQCN1eiT9fHr16nXO/QkJCerdu3elbRdddNF5+42NjbUFdkmKiIhQenq67fO+fft07bXXqkWLFvL19VVsbKwkKTk5uVp1NwWEdgAAAABoZFq1aiWTyaTdu3efdf/u3bsVEBCgkJAQSZKXV90sqOfi4lLps8lkqnTr+5VXXqmMjAy9/fbbWr9+vdavXy+pYjE8VCC0AwAAAEAjExQUpMsvv1yvv/66CgsLK+1LTU3Vxx9/rGuuuUYmk6la/bVt21YbN26stG3Dhg0XVOPJkyeVkJCghx56SIMHD7bdso/KmNMOAAAAALVwNKvw/I0ceJ7XXntNF198sYYNG6Ynn3yy0iPfoqKizjsf/c9uu+02zZw5U/fdd59uvvlmbdmyRXPnzpWkagf/vwoICFBQUJDeeustRUREKDk5Wffff3+t+mrMCO0AAAAAUAMBXq5ydzFr9or99XZOdxezArxca3RM69attXHjRj366KO6+uqrlZGRofDwcI0ePVqPPvroWZ/RXpW4uDh98cUXuvvuu/Xyyy8rPj5eDz74oCZOnCg3N7eaXo4kyWw269NPP9Udd9yhTp06qW3btnrllVc0YMCAWvXXWJms1V2ZoBHLycmRn5+fsrOz5evr6+hyqpayRXqrvzTqJSmo1bnbntwvLZgq3bpKiuxW97UBAAAAjVBRUZESExMVFxcnd3d32/ajWYXKzK+/edcBXq6K8vc4f8N69NRTT2nOnDk6fPiwo0sxrKp+f6Tq51BG2gEAAACghqL8PQwXouva66+/rt69eysoKEirV6/W888/r8mTJzu6rEaP0A4AAAAAOK99+/bpySefVEZGhpo3b667775b06ZNc3RZjR6hHQAAAABwXrNmzdKsWbMcXUaTwyPfAAAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgCO0AAAAAABgUoR0AAAAAAINi9XgAAAAAqKmsw1LByfo7n2eQ5B9df+eDYRDaAQAAAKAmsg5Ls3tLpYX1d04XD2nSBoL7BYiNjdXUqVM1depUu/U5YMAAdevWTS+99JLd+vwrQjsAAAAA1ETByYrAfundkl89hOjsw9IvL1act5qhfc6cObr33nuVmZkpZ+eK2JeXl6eAgABdcsklWrlypa3typUrNXDgQO3fv18tW7asss/T7TIzM+Xv738hV4QaILQDAAAAQG34RUtBrRxdxVkNHDhQeXl52rhxo/r27StJ+uWXXxQeHq7169erqKhI7u7ukqQVK1aoefPm5wzs9mS1WlVeXm77Y0JdKykpkaura72cqy6wEB0AAAAANDJt27ZVRETEGSPqf//73xUXF6d169ZV2j5w4EDNmzdPvXr1ko+Pj8LDw/Wvf/1L6enpkqSkpCQNHDhQkhQQECCTyaQbb7xRkmSxWDRjxgzFxcXJw8NDXbt21RdffFGpf5PJpIULF6pnz55yc3PTr7/+qgEDBmjKlCmaOnWqAgICFBYWprffflv5+fm66aab5OPjo1atWmnhwoW2vsrLy3XzzTfbztW2bVu9/PLLla79xhtv1OjRo/XUU08pMjJSbdu2Pev36J133pG/v7+WLVsmSdqxY4eGDx8ub29vhYWF6frrr9eJEyds7fPz8zVhwgR5e3srIiJCL774Yi1+MjVHaAcAAACARmjgwIFasWKF7fOKFSs0YMAA9e/f37a9sLBQ69ev18CBA1VaWqonnnhCW7du1TfffKOkpCRbMI+OjtaXX34pSUpISNCxY8dsYXnGjBn68MMPNWfOHO3cuVN33nmnrrvuOq1atapSPffff7+eeeYZ7d69W126dJEkffDBBwoODtZvv/2mKVOmaOLEifrnP/+piy++WJs3b9bQoUN1/fXXq6CgQFLFHwiaNWumzz//XLt27dIjjzyiBx54QPPnz690rmXLlikhIUFLlizRggULzvjePPfcc7r//vu1ePFiDR48WFlZWRo0aJC6d++ujRs3atGiRUpLS9PVV19tO+bee+/VqlWr9O2332rx4sVauXKlNm/efCE/omrh9ngAAAAAaIQGDhyoqVOnqqysTIWFhfr999/Vv39/lZaWas6cOZKktWvXqri4WAMHDlTz5s1tx7Zo0UKvvPKKevfurby8PHl7eyswMFCSFBoaapvTXlxcrKefflpLly5VfHy87dhff/1Vb775pvr372/rc/r06br88ssr1di1a1c99NBDkqRp06bpmWeeUXBwsG655RZJ0iOPPKI33nhD27ZtU9++feXi4qLHH3/cdnxcXJzWrl2r+fPnVwrYXl5eeuedd856W/x9992nefPmadWqVerYsaMk6bXXXlP37t319NNP29q99957io6O1t69exUZGal3331XH330kQYPHiyp4g8OzZo1q8mPpFYI7QAAAADQCA0YMED5+fnasGGDMjMz1aZNG4WEhKh///666aabVFRUpJUrV6pFixZq3ry5Nm3apMcee0xbt25VZmamLBaLJCk5OVkdOnQ46zn279+vgoKCM8J4SUmJunfvXmlbr169zjj+9Ii7JDk5OSkoKEidO3e2bQsLC5Mk2236kjR79my99957Sk5OVmFhoUpKStStW7dK/Xbu3Pmsgf3FF19Ufn6+Nm7cqBYtWti2b926VStWrJC3t/cZxxw4cMB2nj59+ti2BwYGVnnrvT0R2gEAAACgEWrVqpWaNWumFStWKDMz0zbqHRkZqejoaK1Zs0YrVqzQoEGDlJ+fr2HDhmnYsGH6+OOPFRISouTkZA0bNkwlJSVVniMvL0+S9MMPPygqKqrSPjc3t0qfvby8zjjexcWl0meTyVRpm8lkkiTbHxA+/fRT3XPPPXrxxRcVHx8vHx8fPf/881q/fv15zyVJl156qX744QfNnz9f999/f6XruPLKK/Xss8+ecUxERIT2799/1v7qA6EdAAAAABqpgQMHauXKlcrMzNS9995r237ZZZdp4cKF+u233zRx4kTt2bNHJ0+e1DPPPKPo6IrHym3cuLFSX6dHrsvLy23bOnToIDc3NyUnJ1e6Fb6urF69WhdffLFuv/1227YDBw5U+/iLLrpIkydP1hVXXCFnZ2fdc889kqQePXroyy+/VGxs7FlXtW/ZsqVcXFy0fv162zSCzMxM7d27t86vm9AOAAAAALWRfdjw5xk4cKAmTZqk0tLSSuGyf//+mjx5skpKSjRw4EA5OzvL1dVVr776qv7v//5PO3bs0BNPPFGpr5iYGJlMJi1YsEAjRoyQh4eHfHx8dM899+jOO++UxWJRv379lJ2drdWrV8vX11c33HBDrWs/m9atW+vDDz/UTz/9pLi4OM2bN08bNmxQXFxctfu4+OKL9eOPP2r48OFydnbW1KlTNWnSJL399tu69tpr9d///leBgYHav3+/Pv30U73zzjvy9vbWzTffrHvvvVdBQUEKDQ3Vgw8+KLO57td2J7QDAAAAQE14BkkuHtIv9fPIL0kV5/MMqvFhAwcOVGFhodq1a2ebHy5VhPbc3Fzbo+Ekae7cuXrggQf0yiuvqEePHnrhhRf0t7/9zXZMVFSUHn/8cd1///266aabNGHCBM2dO1dPPPGEQkJCNGPGDB08eFD+/v7q0aOHHnjggQu/7r+47bbb9Pvvv+uaa66RyWTStddeq9tvv73SY+Gqo1+/fvrhhx80YsQIOTk5acqUKVq9erXuu+8+DR06VMXFxYqJidEVV1xhC+bPP/+87TZ6Hx8f3X333crOzrb7Nf6VyWq1Wuv8LAaXk5MjPz8/ZWdny9fX19HlVC1li/RWf2nUS1JQq3O3PblfWjBVunWVFNmt7msDAAAAGqGioiIlJiYqLi5O7u7uf+zIOiwVnKy/QjyDJP/o+jsf7KLK3x9VP4cy0g4AAAAANeUfTYhGvaj7G/ABAAAAAECtENoBAAAAADAoQjsAAAAAAAZFaAcAAACA82D9btSGPX5vCO0AAAAAUAUXFxdJUkFBgYMrQUN0+vfm9O9RbTh89fijR4/qvvvu08KFC1VQUKBWrVrp/fffV69evSRV/GXi0Ucf1dtvv62srCxdcskleuONN9S6dWtbHxkZGZoyZYq+//57mc1mjR07Vi+//LK8vb0ddVkAAAAAGgEnJyf5+/srPT1dkuTp6SmTyeTgqmB0VqtVBQUFSk9Pl7+/v5ycnGrdl0NDe2Zmpi655BINHDhQCxcuVEhIiPbt26eAgABbm+eee06vvPKKPvjgA8XFxenhhx/WsGHDtGvXLttz7saPH69jx45pyZIlKi0t1U033aRbb71Vn3zyiaMuDQAAAEAjER4eLkm24A5Ul7+/v+33p7YcGtqfffZZRUdH6/3337dti4uLs31ttVr10ksv6aGHHtLf//53SdKHH36osLAwffPNNxo3bpx2796tRYsWacOGDbbR+VdffVUjRozQCy+8oMjIyPq9KAAAAACNislkUkREhEJDQ1VaWuroctBAuLi4XNAI+2kODe3fffedhg0bpn/+859atWqVoqKidPvtt+uWW26RJCUmJio1NVVDhgyxHePn56c+ffpo7dq1GjdunNauXSt/f39bYJekIUOGyGw2a/369frHP/5xxnmLi4tVXFxs+5yTk1OHVwkAAACgMXBycrJLCANqwqEL0R08eNA2P/2nn37SxIkTdccdd+iDDz6QJKWmpkqSwsLCKh0XFhZm25eamqrQ0NBK+52dnRUYGGhr81czZsyQn5+f7RUdHW3vSwMAAAAA4II5NLRbLBb16NFDTz/9tLp3765bb71Vt9xyi+bMmVOn5502bZqys7Ntr8OHD9fp+QAAAAAAqA2HhvaIiAh16NCh0rb27dsrOTlZ0h8LPqSlpVVqk5aWZtsXHh5+xoIQZWVlysjIqHLCv5ubm3x9fSu9AAAAAAAwGoeG9ksuuUQJCQmVtu3du1cxMTGSKhalCw8P17Jly2z7c3JytH79esXHx0uS4uPjlZWVpU2bNtnaLF++XBaLRX369KmHqwAAAAAAoG44dCG6O++8UxdffLGefvppXX311frtt9/01ltv6a233pJUsUrj1KlT9eSTT6p169a2R75FRkZq9OjRkipG5q+44grbbfWlpaWaPHmyxo0bx8rxAAAAAIAGzaGhvXfv3vr66681bdo0TZ8+XXFxcXrppZc0fvx4W5v//ve/ys/P16233qqsrCz169dPixYtsj2jXZI+/vhjTZ48WYMHD5bZbNbYsWP1yiuvOOKSAAAAAACwG5PVarU6ughHy8nJkZ+fn7Kzs409vz1li/RWf2nUS1JQq3O3PblfWjBVunWVFNmt7msDAAAAAFRbdXOoQ+e0AwAAAACAqhHaAQAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgCO0AAAAAABgUoR0AAAAAAIMitAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQRHaAQAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgCO0AAAAAABgUoR0AAAAAAIMitAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQRHaAQAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgCO0AAAAAABgUoR0AAAAAAIMitAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQRHaAQAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgCO0AAAAAABgUoR0AAAAAAIMitAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQRHaAQAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgCO0AAAAAABgUoR0AAAAAAIMitAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQRHaAQAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgCO0AAAAAABgUoR0AAAAAAIMitAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQRHaAQAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgCO0AAAAAABgUoR0AAAAAAIMitAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQRHaAQAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgCO0AAAAAABgUoR0AAAAAAIMitAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQRHaAQAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgCO0AAAAAABgUoR0AAAAAAIMitAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQRHaAQAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgCO0AAAAAABgUoR0AAAAAAIMitAMAAAAAYFAODe2PPfaYTCZTpVe7du1s+4uKijRp0iQFBQXJ29tbY8eOVVpaWqU+kpOTNXLkSHl6eio0NFT33nuvysrK6vtSAAAAAACwO2dHF9CxY0ctXbrU9tnZ+Y+S7rzzTv3www/6/PPP5efnp8mTJ2vMmDFavXq1JKm8vFwjR45UeHi41qxZo2PHjmnChAlycXHR008/Xe/XAgAAAACAPTk8tDs7Oys8PPyM7dnZ2Xr33Xf1ySefaNCgQZKk999/X+3bt9e6devUt29fLV68WLt27dLSpUsVFhambt266YknntB9992nxx57TK6urvV9OQAAAAAA2I3D57Tv27dPkZGRatGihcaPH6/k5GRJ0qZNm1RaWqohQ4bY2rZr107NmzfX2rVrJUlr165V586dFRYWZmszbNgw5eTkaOfOnVWes7i4WDk5OZVeAAAAAAAYjUNDe58+fTR37lwtWrRIb7zxhhITE3XppZcqNzdXqampcnV1lb+/f6VjwsLClJqaKklKTU2tFNhP7z+9ryozZsyQn5+f7RUdHW3fCwMAAAAAwA4cenv88OHDbV936dJFffr0UUxMjObPny8PD486O++0adN011132T7n5OQQ3AEAAAAAhuPw2+P/zN/fX23atNH+/fsVHh6ukpISZWVlVWqTlpZmmwMfHh5+xmrypz+fbZ78aW5ubvL19a30AgAAAADAaAwV2vPy8nTgwAFFRESoZ8+ecnFx0bJly2z7ExISlJycrPj4eElSfHy8tm/frvT0dFubJUuWyNfXVx06dKj3+gEAAAAAsCeH3h5/zz336Morr1RMTIxSUlL06KOPysnJSddee638/Px0880366677lJgYKB8fX01ZcoUxcfHq2/fvpKkoUOHqkOHDrr++uv13HPPKTU1VQ899JAmTZokNzc3R14aAAAAAAAXzKGh/ciRI7r22mt18uRJhYSEqF+/flq3bp1CQkIkSbNmzZLZbNbYsWNVXFysYcOG6fXXX7cd7+TkpAULFmjixImKj4+Xl5eXbrjhBk2fPt1RlwQAAAAAgN04NLR/+umn59zv7u6u2bNna/bs2VW2iYmJ0Y8//mjv0gAAAAAAcDhDzWkHAAAAAAB/ILQDAAAAAGBQhHYAAAAAAAyK0A4AAAAAgEER2gEAAAAAMChCOwAAAAAABkVoBwAAAADAoAjtAAAAAAAYFKEdAAAAAACDIrQDAAAAAGBQhHYAAAAAAAyK0A4AAAAAgEER2gEAAAAAMChCOwAAAAAABkVoBwAAAADAoAjtAAAAAAAYFKEdAAAAAACDIrQDAAAAAGBQhHYAAAAAAAyK0A4AAAAAgEER2gEAAAAAMChCOwAAAAAABkVoBwAAAADAoAjtAAAAAAAYFKEdAAAAAACDIrQDAAAAAGBQhHYAAAAAAAyK0A4AAAAAgEER2gEAAAAAMChCOwAAAAAABkVoBwAAAADAoAjtAAAAAAAYFKEdAAAAAACDIrQDAAAAAGBQhHYAAAAAAAyK0A4AAAAAgEER2gEAAAAAMChCOwAAAAAABkVoBwAAAADAoAjtAAAAAAAYFKEdAAAAAACDIrQDAAAAAGBQhHYAAAAAAAyK0A4AAAAAgEER2gEAAAAAMChCOwAAAAAABkVoBwAAAADAoAjtAAAAAAAYFKEdAAAAAACDIrQDAAAAAGBQhHYAAAAAAAyK0A4AAAAAgEER2gEAAAAAMChCOwAAAAAABkVoBwAAAADAoAjtAAAAAAAYFKEdAAAAAACDIrQDAAAAAGBQhHYAAAAAAAyK0A4AAAAAgEER2gEAAAAAMChCOwAAAAAABkVoBwAAAADAoAjtAAAAAAAYFKEdAAAAAACDIrQDAAAAAGBQhHYAAAAAAAyK0A4AAAAAgEER2gEAAAAAMChCOwAAAAAABkVoBwAAAADAoAjtAAAAAAAYFKEdAAAAAACDIrQDAAAAAGBQhHYAAAAAAAyK0A4AAAAAgEER2gEAAAAAMChCOwAAAAAABmWY0P7MM8/IZDJp6tSptm1FRUWaNGmSgoKC5O3trbFjxyotLa3SccnJyRo5cqQ8PT0VGhqqe++9V2VlZfVcPQAAAAAA9meI0L5hwwa9+eab6tKlS6Xtd955p77//nt9/vnnWrVqlVJSUjRmzBjb/vLyco0cOVIlJSVas2aNPvjgA82dO1ePPPJIfV8CAAAAAAB25/DQnpeXp/Hjx+vtt99WQECAbXt2drbeffddzZw5U4MGDVLPnj31/vvva82aNVq3bp0kafHixdq1a5c++ugjdevWTcOHD9cTTzyh2bNnq6SkxFGXBAAAAACAXTg8tE+aNEkjR47UkCFDKm3ftGmTSktLK21v166dmjdvrrVr10qS1q5dq86dOyssLMzWZtiwYcrJydHOnTurPGdxcbFycnIqvQAAAAAAMBpnR578008/1ebNm7Vhw4Yz9qWmpsrV1VX+/v6VtoeFhSk1NdXW5s+B/fT+0/uqMmPGDD3++OMXWD0AAAAAAHXLYSPthw8f1n/+8x99/PHHcnd3r9dzT5s2TdnZ2bbX4cOH6/X8AAAAAABUh8NC+6ZNm5Senq4ePXrI2dlZzs7OWrVqlV555RU5OzsrLCxMJSUlysrKqnRcWlqawsPDJUnh4eFnrCZ/+vPpNmfj5uYmX1/fSi8AAAAAAIzGYaF98ODB2r59u7Zs2WJ79erVS+PHj7d97eLiomXLltmOSUhIUHJysuLj4yVJ8fHx2r59u9LT021tlixZIl9fX3Xo0KHerwkAAAAAAHty2Jx2Hx8fderUqdI2Ly8vBQUF2bbffPPNuuuuuxQYGChfX19NmTJF8fHx6tu3ryRp6NCh6tChg66//no999xzSk1N1UMPPaRJkybJzc2t3q8JAAAAAAB7cuhCdOcza9Ysmc1mjR07VsXFxRo2bJhef/11234nJyctWLBAEydOVHx8vLy8vHTDDTdo+vTpDqwaAAAAAAD7MFRoX7lyZaXP7u7umj17tmbPnl3lMTExMfrxxx/ruDIAAAAAAOqfw5/TDgAAAAAAzo7QDgAAAACAQRHaAQAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgCO0AAAAAABgUoR0AAAAAAIMitAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQRHaAQAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgCO0AAAAAABgUoR0AAAAAAIMitAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQRHaAQAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgCO0AAAAAABgUoR0AAAAAAIMitAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQRHaAQAAAAAwqFqF9hYtWujkyZNnbM/KylKLFi0uuCgAAAAAAFDL0J6UlKTy8vIzthcXF+vo0aMXXBQAAAAAAJCca9L4u+++s339008/yc/Pz/a5vLxcy5YtU2xsrN2KAwAAAACgKatRaB89erQkyWQy6YYbbqi0z8XFRbGxsXrxxRftVhwAAAAAAE1ZjUK7xWKRJMXFxWnDhg0KDg6uk6IAAAAAAEANQ/tpiYmJ9q4DAAAAAAD8Ra1CuyQtW7ZMy5YtU3p6um0E/rT33nvvggsDAAAAAKCpq1Vof/zxxzV9+nT16tVLERERMplM9q4LAAAAAIAmr1ahfc6cOZo7d66uv/56e9cDAAAAAABOqdVz2ktKSnTxxRfbuxYAAAAAAPAntQrt//73v/XJJ5/YuxYAAAAAAPAntbo9vqioSG+99ZaWLl2qLl26yMXFpdL+mTNn2qU4AAAAAACaslqF9m3btqlbt26SpB07dlTax6J0AAAAAADYR61C+4oVK+xdBwAAAAAA+ItazWkHAAAAAAB1r1Yj7QMHDjznbfDLly+vdUEAAAAAAKBCrUL76fnsp5WWlmrLli3asWOHbrjhBnvUBQAAAABAk1er0D5r1qyzbn/ssceUl5d3QQUBAAAAAIAKdp3Tft111+m9996zZ5cAAAAAADRZdg3ta9eulbu7uz27BAAAAACgyarV7fFjxoyp9NlqterYsWPauHGjHn74YbsUBgAAAABAU1er0O7n51fps9lsVtu2bTV9+nQNHTrULoUBAAAAANDU1Sq0v//++/auAwAAAAAA/EWtQvtpmzZt0u7duyVJHTt2VPfu3e1SFAAAAAAAqGVoT09P17hx47Ry5Ur5+/tLkrKysjRw4EB9+umnCgkJsWeNAAAAAAA0SbVaPX7KlCnKzc3Vzp07lZGRoYyMDO3YsUM5OTm644477F0jAAAAAABNUq1G2hctWqSlS5eqffv2tm0dOnTQ7NmzWYgOAAAAAAA7qdVIu8VikYuLyxnbXVxcZLFYLrgoAAAAAABQy9A+aNAg/ec//1FKSopt29GjR3XnnXdq8ODBdisOAAAAAICmrFah/bXXXlNOTo5iY2PVsmVLtWzZUnFxccrJydGrr75q7xoBAAAAAGiSajWnPTo6Wps3b9bSpUu1Z88eSVL79u01ZMgQuxYHAAAAAEBTVqOR9uXLl6tDhw7KycmRyWTS5ZdfrilTpmjKlCnq3bu3OnbsqF9++aWuagUAAAAAoEmpUWh/6aWXdMstt8jX1/eMfX5+frrttts0c+ZMuxUHAAAAAEBTVqPQvnXrVl1xxRVV7h86dKg2bdp0wUUBAAAAAIAahva0tLSzPurtNGdnZx0/fvyCiwIAAAAAADUM7VFRUdqxY0eV+7dt26aIiIgLLgoAAAAAANQwtI8YMUIPP/ywioqKzthXWFioRx99VKNGjbJbcQAAAAAANGU1euTbQw89pK+++kpt2rTR5MmT1bZtW0nSnj17NHv2bJWXl+vBBx+sk0IBAAAAAGhqahTaw8LCtGbNGk2cOFHTpk2T1WqVJJlMJg0bNkyzZ89WWFhYnRQKAAAAAEBTU6PQLkkxMTH68ccflZmZqf3798tqtap169YKCAioi/oAAAAAAGiyahzaTwsICFDv3r3tWQsAAAAAAPiTGi1EBwAAAAAA6g+hHQAAAAAAgyK0AwAAAABgUIR2AAAAAAAMitAOAAAAAIBBEdoBAAAAADAoQjsAAAAAAAZFaAcAAAAAwKAI7QAAAAAAGBShHQAAAAAAgyK0AwAAAABgUA4N7W+88Ya6dOkiX19f+fr6Kj4+XgsXLrTtLyoq0qRJkxQUFCRvb2+NHTtWaWlplfpITk7WyJEj5enpqdDQUN17770qKyur70sBAAAAAMDuHBramzVrpmeeeUabNm3Sxo0bNWjQIP3973/Xzp07JUl33nmnvv/+e33++edatWqVUlJSNGbMGNvx5eXlGjlypEpKSrRmzRp98MEHmjt3rh555BFHXRIAAAAAAHZjslqtVkcX8WeBgYF6/vnnddVVVykkJESffPKJrrrqKknSnj171L59e61du1Z9+/bVwoULNWrUKKWkpCgsLEySNGfOHN133306fvy4XF1dq3XOnJwc+fn5KTs7W76+vnV2bRcsZYv0Vn9p1EtSUKtztz25X1owVbp1lRTZre5rAwAAAABUW3VzqGHmtJeXl+vTTz9Vfn6+4uPjtWnTJpWWlmrIkCG2Nu3atVPz5s21du1aSdLatWvVuXNnW2CXpGHDhiknJ8c2Wn82xcXFysnJqfQCAAAAAMBoHB7at2/fLm9vb7m5uen//u//9PXXX6tDhw5KTU2Vq6ur/P39K7UPCwtTamqqJCk1NbVSYD+9//S+qsyYMUN+fn62V3R0tH0vCgAAAAAAO3B4aG/btq22bNmi9evXa+LEibrhhhu0a9euOj3ntGnTlJ2dbXsdPny4Ts8HAAAAAEBtODu6AFdXV7VqVTE/u2fPntqwYYNefvllXXPNNSopKVFWVlal0fa0tDSFh4dLksLDw/Xbb79V6u/06vKn25yNm5ub3Nzc7HwlAAAAAADYl8NH2v/KYrGouLhYPXv2lIuLi5YtW2bbl5CQoOTkZMXHx0uS4uPjtX37dqWnp9vaLFmyRL6+vurQoUO91w4AAAAAgD05dKR92rRpGj58uJo3b67c3Fx98sknWrlypX766Sf5+fnp5ptv1l133aXAwED5+vpqypQpio+PV9++fSVJQ4cOVYcOHXT99dfrueeeU2pqqh566CFNmjSJkXQAAAAAQIPn0NCenp6uCRMm6NixY/Lz81OXLl30008/6fLLL5ckzZo1S2azWWPHjlVxcbGGDRum119/3Xa8k5OTFixYoIkTJyo+Pl5eXl664YYbNH36dEddEgAAAAAAduPQ0P7uu++ec7+7u7tmz56t2bNnV9kmJiZGP/74o71LAwAAAADA4Qw3px0AAAAAAFQgtAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQRHaAQAAAAAwKEI7AAAAAAAGRWgHAAAAAMCgCO0AAAAAABgUoR0AAAAAAIMitAMAAAAAYFCEdgAAAAAADIrQDgAAAACAQRHaAQAAAAAwKEI7AAAAAAAGRWhvTKxWqbzU0VUAAAAAAOzE2dEFwI52fSttfEcK7yxF9nR0NQAAAACAC0Robyws5dLOLyu+Tt1e8ZKkzR9KQa0kN2/H1QYAAAAAqBVuj28sjmyUCjMldz+p8zWS66mQvvFd6fs7HFsbAAAAAKBWCO2Nxf7FFe8tBkk9rpcGP/bHvl3fSYVZjqgKAAAAAHABuD2+gUm0hCs/07nidvhTnIsz1fbIBpkk7QserOLj5VKOu7ws4YoL8pAyE6U9P0jdxzuucAAAAABAjRHaG5DErDINLJkpLZOkfNv225x+0jQXizZZWmvs4sBT+wIlzdTvkYsVkJko7fyK0A4AAAAADQyhvQHJL7FIkia1zVdUaFDFRqtVo3askoqk0haX6+kQd0nS0fSTmp3gpeMR/RWwc650cKVUkCF5BjqmeAAAAABAjTGnvQGK8ixXnL9Zcf5mdVGCfItSVO7kLlPcpbbtUZ4Vt8+XeDereAScpUza/Z2DKwcAAAAA1AShvYELTVkqSToZdokszh5nb9RxTMX7jq/qqSoAAAAAgD0Q2hswc1mhglJXS5LSo4ZU3bDjPyrek36R8tLroTIAAAAAgD0wp70BC079VU6WYhV6RinXr93ZG2UlS6GuUkg76fgeae3sP0L8aW4+UlDLui8YAAAAAFAjhPYGLOTYckmnRtlNpso7nV0q3pc/Ia1M+mP76pcqXn81ZTPBHQAAAAAMhtDeQJksZfLO3idJOhkaf2YD9wBJRdKl90gBZVJhprTscUkmachjkrtfRbvsw9IvL0rFufVVOgAAAACgmgjtDZR7wVGZrWUqc/JQsUdY1Q39oqUgp4qvQ9pLx3dL2UekqJ71UygAAAAAoNZYiK6B8spNkiQV+MSeeWt8VWL7Vbwn/VInNQEAAAAA7IvQ3kB55iVJkgq8Y6t/UMwlFe/HE6SSArvXBAAAAACwL0J7A+WZe0iSlO8TW/2DvIIlr1BJVunkvjqpCwAAAABgP4T2BsorN1GSVOAdU7MDQ9pWvB9PsHNFAAAAAAB7I7Q3QG5luXItyZRUi9AefCq0nyC0AwAAAIDREdobIP/io5KkIo9wWZw9anbwn0farVY7VwYAAAAAsCdCewPkX3REUg3ns58W2EIyO0tFWVJ+ul3rAgAAAADYF6G9ATo90l6jleNPc3aTAuIqvmZeOwAAAAAYGqG9AQo4NdJeUJuRdumPW+SZ1w4AAAAAhkZob2CcVC6/kmOSpPzajLRLfyxGx0g7AAAAABgaob2BaWE6Jidrmcqd3FXsEVq7TkLaVLyfPCBZyuxXHAAAAADArgjtDUx7U7KkU6Psplr++HwiJTcfyVIq5aTYrzgAAAAAgF0R2huYduaK0F7r+eySZDJJwadG2zMPXXhRAAAAAIA6QWhvYNqbKkJ2gXfMhXUU0q7iPYvQDgAAAABGRWhvYNqZD0uS8n3iLqwj20h70oX1AwAAAACoM4T2BsSpJEcRpgxJdhhpP72CfMGJC6wKAAAAAFBXCO0NiHv2QUlSrkuwLM4eF9aZm7fk28wOVQEAAAAA6gqhvQE5Hdqz3KLs0+HpR78BAAAAAAyJ0N6AuOecCu3udhohD25nn34AAAAAAHWC0N6AnA7tmW52Cu1/Hmm3WuzTJwAAAADAbgjtDUV5mdxykiRJWe52uj0+IFYyu1R8nX3EPn0CAAAAAOyG0N5QZByQ2VKqfKub8lyC7dOn2VnyO/UHgBP77NMnAAAAAMBuCO0NhXeYkns9qBll/5JMdvyx+UZWvGccsF+fAAAAAAC7ILQ3FB7+ymk2QB+VX27ffn1PjbSfZKQdAAAAAIyG0N7UnQ7tJ/Y7tg4AAAAAwBkI7U3d6dvjCzOkvHTH1gIAAAAAqITQ3tQ5uf7xdep2x9UBAAAAADgDoR1/ILQDAAAAgKEQ2vEHQjsAAAAAGAqhHX8gtAMAAACAoRDa8YeT+6TSQkdXAQAAAAA4hdCOCh4BktUipe9ydCUAAAAAgFMI7agQ1KrinVvkAQAAAMAwCO2oQGgHAAAAAMMhtKMCoR0AAAAADMfZ0QWgbu3Pspy7QbazvCzhirOF9h2SxSKZ+XsOAAAAADgaob2Rcj/1k526/HyrwQdKmqkV1mDFObtLpflSZqIU1LKuSwQAAAAAnAehvZGK8DZr5kB3FZWdu93R9JOaneCl/DKTFNpBStlccYs8oR0AAAAAHI7Q3ohFeFfjFve88j++Du/8R2jvOLrO6gIAAAAAVA8Tl/GH8M4V7yxGBwAAAACGQGjHHwjtAAAAAGAohHb8IaxjxXtuipR/0rG1AAAAAAAI7fgTNx8psEXF12mMtgMAAACAoxHaURm3yAMAAACAYRDaUVkYoR0AAAAAjILQjspsI+07HFsHAAAAAIDQjr84HdpPJEilRY6tBQAAAACaOEI7KvONlDwCJUuZdHyPo6sBAAAAgCaN0I7KTCYpvFPF18xrBwAAAACHIrTjTOFdKt4J7QAAAADgUA4N7TNmzFDv3r3l4+Oj0NBQjR49WgkJCZXaFBUVadKkSQoKCpK3t7fGjh2rtLS0Sm2Sk5M1cuRIeXp6KjQ0VPfee6/Kysrq81Ial9Pz2tNYjA4AAAAAHMmhoX3VqlWaNGmS1q1bpyVLlqi0tFRDhw5Vfn6+rc2dd96p77//Xp9//rlWrVqllJQUjRkzxra/vLxcI0eOVElJidasWaMPPvhAc+fO1SOPPOKIS2oc/vysdqvVsbUAAAAAQBPm7MiTL1q0qNLnuXPnKjQ0VJs2bdJll12m7Oxsvfvuu/rkk080aNAgSdL777+v9u3ba926derbt68WL16sXbt2aenSpQoLC1O3bt30xBNP6L777tNjjz0mV1dXR1xawxbcRnJylYpzpKxDUkCsoysCAAAAgCbJUHPas7OzJUmBgYGSpE2bNqm0tFRDhgyxtWnXrp2aN2+utWvXSpLWrl2rzp07KywszNZm2LBhysnJ0c6dO896nuLiYuXk5FR64U+cXKSQdhVfM68dAAAAABzGoSPtf2axWDR16lRdcskl6tSpYvXy1NRUubq6yt/fv1LbsLAwpaam2tr8ObCf3n9639nMmDFDjz/+uJ2voGHbn1EmHc22fY7ybaeA1G1K37dB6b6X2bZ7uTkrLtjLESUCAAAAQJNjmNA+adIk7dixQ7/++mudn2vatGm66667bJ9zcnIUHR1d5+c1InfnijnrUxdnSov/+N7f6OSux1ykLRt+1a1r+lQ6ZsU9AwjuAAAAAFAPDBHaJ0+erAULFujnn39Ws2bNbNvDw8NVUlKirKysSqPtaWlpCg8Pt7X57bffKvV3enX5023+ys3NTW5ubna+ioYpwsOimc6zVXTpg5L/H3+4CMkoltZ/qIu9j+npARUL0x3NKtTsFfuVX8zK/AAAAABQHxw6p91qtWry5Mn6+uuvtXz5csXFxVXa37NnT7m4uGjZsmW2bQkJCUpOTlZ8fLwkKT4+Xtu3b1d6erqtzZIlS+Tr66sOHTrUz4U0cBGmDMUFVNz2fvrlF9tdkuRdmKJWPmWKC/ZSlL+HgysFAAAAgKbFoSPtkyZN0ieffKJvv/1WPj4+tjnofn5+8vDwkJ+fn26++WbdddddCgwMlK+vr6ZMmaL4+Hj17dtXkjR06FB16NBB119/vZ577jmlpqbqoYce0qRJkxhNvwDlrr4q8mom9/wj8srcrZzwvo4uCQAAAACaHIeOtL/xxhvKzs7WgAEDFBERYXt99tlntjazZs3SqFGjNHbsWF122WUKDw/XV199Zdvv5OSkBQsWyMnJSfHx8bruuus0YcIETZ8+3RGX1KgUBFbcqeCZudvBlQAAAABA0+TQkXar1XreNu7u7po9e7Zmz55dZZuYmBj9+OOP9iwNkvID2ivw8GJ5Ze5ydCkAAAAA0CQZ6jntMJb8gPaSJK8MRtoBAAAAwBEI7ajS6dvjPbL3yVRe4uBqAAAAAKDpIbSjSsVeUSpz8ZHZUiqP7AOOLgcAAAAAmhxCO6pmMimfxegAAAAAwGEI7TingtPz2lmMDgAAAADqHaEd53R6pN0rY6eDKwEAAACApofQjnPKC+wsSfI+uUMma7mDqwEAAACApoXQjnMq9GulcmdPOZXlyycvydHlAAAAAECTQmjHuZmdlB/YUZIUlL3dwcUAAAAAQNNCaMd55QV1kSQFZe9wcCUAAAAA0LQQ2nFeecFdJUlB2SxGBwAAAAD1idCO8zo90u6fkyBXlTq4GgAAAABoOgjtOK9i72iVugXIyVqqdqZkR5cDAAAAAE0GoR3nZzIpP6ji0W9dzAcdXAwAAAAANB2EdlRLXlDFvPaupgMOrgQAAAAAmg5nRxcAg8g+fM7dea7BkipG2pnVDgAAAAD1g9De1Dm7Vbz//MI5m+WpTHKSWpuOas+JBCnqonooDgAAAACaNkJ7U+cVLF16t1RWfM5mpZLyNz0gr/JsuR/fKonQDgAAAAB1jdCOiuBeDRmecfLK3SKPzL11XBAAAAAAQGIhOtTASfdYSZJnVoJjCwEAAACAJoLQjmo76RErSfLIJLQDAAAAQH0gtKPaMtybS5JcC1Kl/BMOrgYAAAAAGj/mtDdxm9PKte14ucotUrnVqnKL5Opk0uAYZ8X6Vf6bTqmTpw5YItTSfExK+V1qfbmDqgYAAACApoHQ3kQVlVn14c4SrUguP+v+ZYfKNKC5k65u5yp/N5Nt+1ZrS7XUMenoZkI7AAAAANQxQnsTdCjbolc2FyslzyqTpAHNnRTkYZaTSTKbpMRsi9allGtFcrnWpRTqH61ddEVcxa/KNksLjXH6VUrZ7NiLAAAAAIAmgNDexCxOKtW8naUqs0gB7ibd3t1VnYKdzmh3RVy5PtxRqoPZFn2yu1S/p5frmqiK0C5JOrpJslolk+mMYwEAAAAA9kFob0I2pZbp/e2lkqQeYU66raurfN3OHrrbBjrpiUvN+uVIuT7YUaLdJy36sNhTR6yxsphdZM4/Lp3cLwW3rs9LAAAAAIAmhdXjm4i8Eqve2VYiSRoW56x7elcd2E8zm0zqH+2saX3d5OEsHcxzVolclenfuaJB0q91XTYAAAAANGmE9ibigx0lyiqWIr1M+ld7F5lqcFt76wAn3dfHTa5mqyRpUu4NKrK6ENoBAAAAoI4R2puAjall+vVouUyS/q+7q1ydaj4PvW2gk25oWSBJWpcbosmld8iStKZiXjsAAAAAoE4Q2hu5vBKr3j11W/yols5qHXDmonPVFedd8Xg4V7O01NJTr2b1kTIO2qVOAAAAAMCZCO2N3NxTt8VHeZt0VVsXu/Q5qbePJOmlsrH6Zf16u/QJAAAAADgTob0R23a8XKtP3xbfrXa3xZ/N4Dh3XRt1QlaZ9Z817krJKrRLvwAAAACAygjtjdj3+yse7zY0zlmtLuC2+LN5dGi0OpoSlVHmpsmfbFZJmcWu/QMAAAAACO2NVnKORTtOWGSSNLKFs937d4/rozfcZstH+dqcnKUZC3fb/RwAAAAA0NQR2hupHw9WjLJfFOGkEM86+DG7eql5dLRmurwhSXp/dZKW7kqz/3kAAAAAoAkjtDdCWcVWrT5asdL7yJb2H2W3iblElztt1r/DD0iS7v9qm07mFdfd+QAAAACgiSG0N0JLkkpVZpFaB5gv6BFv5xXbT5J0j+U9tQ3z0Ym8Ek37arusPLsdAAAAAOyC0N7IlJRbtTSpTJI0og7mslcS3UcyOck9J1Ezh4fKxcmkxbvS9PmmI3V7XgAAAABoIgjtjcyvR8qVUyIFe5jUO7wOR9klyc1biuohSepYtFl3Xt5GkjT9+106nFFQt+cGAAAAgCaA0N6IWK1WLUysWIBuWJyznMz2eS77OcVcUvGe9Ktuu6ylesUEKK+4THfP36pyC7fJAwAAAMCFILQ3ItuPW3Qk1yp3J2lQ8zq+Nf60U/PalfSrnMwmzby6m7xcnfRbUobe+eVg/dQAAAAAAI0Uob0RWXqoYi77gObO8nSph1F2yTavXVmHpKxkNQ/y1MOjOkiSXly8V7uP5dRPHQAAAADQCBHaG4mCUqu2pFc85m1AfY2yS5K7r9SsV8XXe3+SJF3TO1pD2oeqpNyiOz/bouKy8vqrBwAAAAAaEUJ7I7EptVylFinS26TmPnU7yr4/o0w7jmbbXqnhAyVJuVu/046j2dqZkqMbLo6Vv4eL9qTmauaSvXVaDwAAAAA0VvU4JIu6tCal4tb4+EhnmUx1E9rdnSsWlpu6OFNa/Ktte0tTkJa5SW5HVuvaVxcrV56Vjnvr54Ma3C5MF8UF1kldAAAAANBYEdobgdwSq7Yft0iS4iPr7jFvER4WzXSeraJLH5T8o/+0p7Nyfp4t3/wkvdE3U8kRfXQ0q1CzV+zXkPahWro7XXfN36KF/7lUPu4udVYfAAAAADQ23B7fCGw4Vq5yqxTja1KUT93+SCNMGYoLcFZcsFelV27MUElSu6yfFRfspSh/D0nSrZe1ULMADx3JLNQTC3bVaW0AAAAA0NgQ2huBtX+6Nd5RMqOHSJL8j66UqbzEtt3T1Vkv/rOrTCZp/sYjWrwz1VElAgAAAECDQ2hv4LKKrdp54tSt8VF1d2v8+eQGd1eJe5CcS3Plm7a+0r4+LYJ066UtJEnTvtquE3nFjigRAAAAABocQnsD91tKmaySWvqbFerpwB+n2UmZzQZLkgKPLD1j911D26hduI9O5pfo/i+3y2q11neFAAAAANDgENobuDUpFc9Av7gOF6Crrsxml0uSAg4vlf4Syt2cnTTrmm5ydTJr6e40zd942BElAgAAAECDQmhvwE4WWpSQUXFrfB8DhPbsiEtU7uQht4JjCsjZfcb+9hG+umtoG0nS9O93KflkQX2XCAAAAAANCqG9AVt3apS9XaBZQR6O/1FanN2VHXmpJKlZ+oqztrnl0ha6KDZQ+SXlumv+FpVbuE0eAAAAAKri+KSHWlt7KrQ7cgG6v8o4tYp8s7Szh3Yns0kvXt1VXq5O2ngoU2/9fLA+ywMAAACABoXQ3kBlFVl1IKvi1vje4Y571NtfZUYNktVkVkBugpqZjp+1TXSgpx69sqMkaeaSBO1Mya7PEgEAAACgwSC0N1Bb0itG2eP8zApwNzm4mj+UuQcqN6SXJOkK829Vtvtnr2a6vEOYSsutuuuzrSoqLa+vEgEAAACgwSC0N1C/nwrt3cOM9yM8EXelJGmc04ozVpE/zWQyacaYzgr2dlVCWq5mLtlbnyUCAAAAQINgvMSH8yqzSNuPnwrtocaZz37aidi/qczJQ63MKfJM21Blu2BvN80Y00WS9PYvB7Xu4Mn6KhEAAAAAGgRCewN0KN9JhWWSr6vUwt94P8JyVx8dirhCkhSw53/nbHt5hzBd0ytaVqt09/ytyi4orY8SAQAAAKBBMF7iw3klZFcsPNct1ElmkwPms2cflk7uP+drv3/Fo9/8Di6QCjPP2d3DV3ZQdKCHjmYV6q75W2ThMXAAAAAAIEkyzrLjqLY9ORU/tu5h9XxrvLNbxfvPL5y36UlrmHabo9Veh6Vtn0t9bq2yrbebs94Y31Nj31ijZXvS9ery/frPkNb2qhoAAAAAGixCewN0othJTiapS0g9h3avYOnSu6Wy4vO3PZ6tT/cm6HHzB9KmudJFt0jnuCugU5SfnhzdSfd+sU0vLdurLs38NLBdqP1qBwAAAIAGiNvjG6i2gWZ5ujjg1nivYMkv6vwv90B9XX6JLGZXKX2ndHTTebv+Z69oje/TXFar9J9Pf9ehk/n1cEEAAAAAYFyE9gaqmwFXjf+rHHkrO6p/xYdN71frmEeu7KBu0f7KKSrTbfM2qbCE57cDAAAAaLoI7Q1IUdkfC7TV+3z2WtoeOFSSZNn+pXYlHtGOo9lnvBJP/DGi7ubspDeu66Fgb1ftSc3V3Z+zMB0AAACApos57Q3I1rQSSVKAq0VR3g64Nb4G3J0rgvaE9ZFa4hql1mVH9fE7L+rj8iFnbb/ingGKC/aSJEX4eej18T01/p11+nF7qp4PStB9V7Srt9oBAAAAwCgI7Q3IhpSK0N7Wt0wmRzzqrQYiPCya6TxbRZc+qJzM8dKe53Sf/3J16jdFVvMfv3ZHswo1e8V+5ReXVTr+orhAPTOmi+7+fKveWHlAcUFeurp3dH1fBgAAAAA4FLfHNxBWq1Ub/xTaG4IIU4biApxV1u06lboFyDc/Sb1zlyou2Mv2ivL3qPL4sT2b6Y5BrSRJD3y9XWv2n6iv0gEAAADAEAjtDcTuY7k6UWiRJLXwaRih/TSLi7dSOlY8p73ZtldlspRW+9g7L2+jK7tGqsxi1f99tEn70/PqqkwAAAAAMBxCewMR6e+uyb28JUkuDfCnltrmepW4B8k9L1khB748Y//+9LyzLlK3MyVHN10cq3bhPsopKtP4t9fpWHahA64AAAAAAOofc9obCH9PV13RykOvbWyYI80WF08d7TRRcRufVLNtr+p4i3/I6uQm91N/gZj62ZZq9ZOWW6xxb63TN7dfogAv1zqsGAAAAAAcj9COepPWZryidr4tt4JjCtv3mVLbTVCEn4dmXt1VRaWW8x6/OzVH89Ye0qGTBbpp7gZ9/O8+8nLjVxgAAABA40XiQb2xOrnpSOdJavHbI4ra8brSW10ti7O7IvyqXozubHzcnbXlcJb+76NNeueGXnJzbhjPrAcAAACAmmqAs6PRkKW3ulpFXlFyLUxX2N6PatXHY1d2lKerk37Zd0J3frZFZeXnH6UHAAAAgIaI0I56ZXVy1dHOkyVJUTvekHPRyRr30TbcR29e31MuTib9uD1Vd83fqnKL1d6lAgAAAIDDEdpR7463HKN8/7ZyKc5U3Ibpterj0tYhmv2vHnI2m/Td1hTd8znBHQAAAEDjQ2hHvbOaXXTg4mdlNZkVnPS9Ag4vrVU/QzuG67V/9ZCT2aSvfz+q/36xTRaCOwAAAIBGhNAOh8gP6qKUDv+WJLVY/5CcSnJq1c8VncL1yrjucjKb9OXmI7r/K4I7AAAAgMbDoaH9559/1pVXXqnIyEiZTCZ98803lfZbrVY98sgjioiIkIeHh4YMGaJ9+/ZVapORkaHx48fL19dX/v7+uvnmm5WX1zCfZd7UHOkyVYU+sXItTFfMpqdr3c/ILhGadU03mU3S/I1HdM/nW1mcDgAAAECj4NDQnp+fr65du2r27Nln3f/cc8/plVde0Zw5c7R+/Xp5eXlp2LBhKioqsrUZP368du7cqSVLlmjBggX6+eefdeutt9bXJeACWJzddSD+GUlS2P758jv2a7WO25+epx1Hsyu9WgR76e6hbWU2SV/9flQ3vP+b9qbl1mX5AAAAAFDnHPqc9uHDh2v48OFn3We1WvXSSy/poYce0t///ndJ0ocffqiwsDB98803GjdunHbv3q1FixZpw4YN6tWrlyTp1Vdf1YgRI/TCCy8oMjKy3q4FtZMbdpGOtZ2giIQP1WLtA9o2aoHKXX3P2tbdpeJvTFM/23LeflfvP6mhs37W6+N7qHmg53nbe7k5Ky7Yq0a1AwAAAEBdc2hoP5fExESlpqZqyJAhtm1+fn7q06eP1q5dq3Hjxmnt2rXy9/e3BXZJGjJkiMxms9avX69//OMfZ+27uLhYxcXFts85ObWbTw37SO5+rwKOLJN7/hG1Wn23Ega8KZnOvAkkws9DM6/uqqLSc9/6vjctVx+tO6Qyi1W3f7y52nWsuGcAwR0AAACAoRg2tKempkqSwsLCKm0PCwuz7UtNTVVoaGil/c7OzgoMDLS1OZsZM2bo8ccft3PFOKvsw+dtYnHx0N7LXlOnn65R4JFlarbtZR3peudZ20b4eZy3v7hgLzUL8NBzPyWopMyi6AAPTYiPlZfb2X/dj2YVavaK/covLjtv3wAAAABQnwwb2uvStGnTdNddd9k+5+TkKDo62oEVNULObhXvP79Qreb5gx7WgU7/Uettzyt626sqcA5QRni/Mxu6eEi+Ueftr2Oknx4c0V7P/bRHhzML9f7qRN0/vL1CfNxqchUAAAAA4FCGDe3h4eGSpLS0NEVERNi2p6WlqVu3brY26enplY4rKytTRkaG7fizcXNzk5sb4a1OeQVLl94tlRWfu11RjvT7h9LyJ3RCkrcpQBHmTLXa9IS2W2JVKPczjxnzVrWCe5swHz12ZUfNWLhHKdlFevS7Hbp/ePtqzXEHAAAAACMwbGiPi4tTeHi4li1bZgvpOTk5Wr9+vSZOnChJio+PV1ZWljZt2qSePXtKkpYvXy6LxaI+ffo4qnSc5hV8/jZ+UZXCfZK1XJ4Jr8kvZ6/aeeZqe8eJKnM+Nc88P13aNl8qLax2Cc0CPDX9bx31zKI9OpJZqMe/36l7hrZV+4izL3YHAAAAAEbi0Ee+5eXlacuWLdqyZYukisXntmzZouTkZJlMJk2dOlVPPvmkvvvuO23fvl0TJkxQZGSkRo8eLUlq3769rrjiCt1yyy367bfftHr1ak2ePFnjxo1j5fiGxCu4Irz7RUn+zbW3+0Mqcg+Ve/EJtTvwrsxegRX7vELP39dZBHm76dErO6ptmI8KSso1Y+Fu/ZaYYeeLAAAAAAD7c2ho37hxo7p3767u3btLku666y51795djzzyiCTpv//9r6ZMmaJbb71VvXv3Vl5enhYtWiR39z9umf7444/Vrl07DR48WCNGjFC/fv301ltvOeR6YB9lrr7a0/1Blbr4yCc7Qe22PCVz+Xlusz8PbzdnPTCivXrFBKi03KqXlu3Vkl1pdqoYAAAAAOqGyWq1Wh1dhKPl5OTIz89P2dnZ8vU17m3TO7b8plGfHtfT3XMU16zqOfuNhVf2PnXY9IicywuVGdxTCbETZF0/R7ryZSmoVa36LLdY9f7qRC3bU7EWwtgeUerRPEAPfrNDC6b0U6coP3teAgAAAACcVXVzqENH2oFzyfdrrT3dH1K52VUBJzap1cG5ki7sb0xOZpNu7hensT0qFrL7cvNRfbPl6IUXCwAAAAB1gNAOQ8sN6KiErtNkMTkrOON3tTKlyGS5sOepm0wmXdUzWv/vkjiZJG1Iyqw4V1GpHSoGAAAAAPshtMPwsoO7a1+Xe2QxmRVizlGbzdNlLiu64H4v7xCmu4e2latzxT+Duz/fqoPH8y64XwAAAACwF0I7GoSM0L5KaH2bLFaTAtPXqd3ym+RUknvB/faMCdBtl7WQJKVkFekfr6/Rmv0nLrhfAAAAALAHwz6nHfirLP9O2mVprnZux+WXtl4dll6n3YPeV5l74AX1G+HnIUmKDfJU0skCXf/eb/q//i00vFPEGW293JwVF+x1QecDAAAAgOoitKNByZWndvV5Xu03Pizvk9vV6aertXvQeyr2aV7rPt1dKm44STpZIKlihfnZKw5o9ooDZ22/4p4BBHcAAAAA9YLQjgYn36+1dgz7TB2WTpBHzkF1XjhWewa+rbyQbrXqL8LPQzOv7qqiUousVqtW7j1ue4Z7mzBvjevdXO4uTjqaVajZK/Yrv/jCFsIDAAAAgOpiTjsapCK/lto+/CvlBXaUS/FJdVxyrQIPLax1fxF+HooL9lKLEG/9v0viNHVwa7k6mbU3LU/v/pooT1cnRfl72PEKAAAAAOD8CO1osEo9w7Rz6KfKiBokc3mx2vw8WRE735asF/Ysd0nq0yJIj1zZQQGeLjqaVagHv96uhNQcO1QNAAAAANVHaEeDZnHxUsKAN3Ws7QSZZFXs5hlquea/MpUXX3DfLUO89eTozmod6q38knJ9uPZQxTnt8EcBAAAAAKgOQjsaPrOTkno/qsReD8tqMiv04JfquPhauRSkXXDXgV6uenhUBw1pH6bTUf2JBbuUXVh6wX0DAAAAwPkQ2tE4mExKbX9TxSPgXP3kc2KLuvz4d3kf33LBXbs4mXVzvziN7dFMkrQhKVNXvvqrth3JuuC+AQAAAOBcCO1oVLIjL9X24V+rwK+1XAvT1XHxOAUf+MouffeMCZAkhfm6KTmjQGPfWKO5qxNl5XZ5AAAAAHWER76h4ck+fM7dRZK293lerbc+q8C0tWq95h55pazRoXa3SGansx/k4iH5RlXr9C+P6673Vyfqp51peuz7XVp3MEPPXtVFfh4uNbwQAAAAADg3QjsaDme3ivefXzhvU4ukBFkVbQpWM/MJRSZ9Jc/En7TPEqUyVRHcx7xVreDu7easOdf11Nw1SXr6x91atDNV249ma9Y13XRRXGANLggAAAAAzo3QjobDK1i69G6prPorwx+WlJ+xWa0OzpO/JV+dPTO0p/VtKvSM/KNRfrq0bb5UWljtfk0mk266JE49mgdoyv9+V3JGgca9tVYTB7TU1CFt5OLEzBMAAAAAF47QjobFK7jGh2T4RWlHcCe13fK03IuOq/PuF7Wv013KDL3ogsvpGu2vH+7op8e/36UvNh3R7BUH9Ou+E5p1TTe1CPG+4P4BAAAANG0MB6JJKPCJ0/Y+Lyg7oKOcyovUbuvTijo4X7LDInI+7i564Z9d9fr4HvLzcNHWI9ka+cqven91oiwWFqkDAAAAUHuMtKPJKHP10+4ejyt273sKP/yjmh/4RJ55STrQ7CpZ7ND/iM4R6t7cX/d8vlWr95+0jb7fMbi1ovw9znmsl5uz4oK97FAFAAAAgMaE0I4mxWp2VmK7W5XvHau4PW8pOG2NPHIOKUHuqv5M+apF+Hlo3v/ro1eX79Ospfu0MyVHt83bVK1jV9wzgOAOAAAAoBJCO5qk9GZDVejVTG22PSuvwqPqbHbS3hO/Kyeo1XmP3Z+ed942MUEV4btZgIeOZBbavh7dLUqRfxl1P5pVqNkr9iu/uKwWVwIAAACgMSO0o8nKDeig7X1eUNtNT8i7IFkdNkxTkiVPqe1ulEymM9q7u1QsATH1sy3VPsedQ1pr97Fcfbw+WUcyCzV75X4N7xiuf/aKlrtLFY+eAwAAAIBTCO1o0krcQ7Szw51q8dujCjHnKG7jE/I6tk4HO/1HVifXSm0jJM0cEa4it5Bq9e3uYlaEn4ci/T3VIyZA89Ye0tqDJ/XjjlStS8zQhPgYXRTLc90BAAAAVI3QjibP4uqj/dZI5VvcFWNKV+jRJfI88osSLM1UIpdKbSMkacxbkm9Ujc4R4OmqOwa3Vv82IXpvdaLSc4v10tJ96hjpq8Htw+x3MQAAAAAaFUI74BUsXXqPjpUVqyB7j1rvf0/e5fnq7J6mva1uUa5Py4p2+enStvlSaWGtT9U12l/PXdVF321J0ffbUrQzJUe7j+VIkvKKmNMOAAAAoDKe0w5IFcHdL0rZzQdre98Xle8dI9fSXHXY84pCc3dIflGSV6hdTuXm7KR/9orWC1d11UWxgTr9KPd/f7hRb/18QEWl5XY5DwAAAICGj9AO/EWxZ7h29H5GJ0Mvltlappa731Dc7jkyWew7Eh7q6647L2+j/3dJnCQpr7hMT/+4R4NeWKnPNx5W+ek0DwAAAKDJIrQDZ2Fx9tDeLvcqudV4WWVS+JFF6rDnFbnI/rewtwr1liRNHdJakX7uSsku0r1fbNPwl3/W0l1psloJ7wAAAEBTRWgHqmIy6WjcP7Wn24Mqc/aUb94BdTYnyitrb52cbkj7MC2/Z4AeHNFefh4u2puWp39/uFFXv7lWmw5l1Mk5AQAAABgboR04j6yQXtp+0fMqdA+Tm6lMndbdqeCDX9fJudxdnHTLZS30838HauKAlnJzNmtDUqbGvrFWt3y4UbtScurkvAAAAACMidAOVEORV5S2d7hXGVZvmS2lar36bsVsfFKy8zz30/w8XHTfFe206t6BGtc7WmaTtGRXmka88otu/XCjdhzNrpPzAgAAADAWHvkGVFO5s4cSLM0UHdNKzY4sUOTu9+SZ/rv2dX9QZa6+Zx7g4lGj57nvT8876/br+saof5sQ/e+3ZP2y74QW70rT4l1pGtI+VFMGtVbXaP9aXhEAAAAAoyO0A9Xl7CbJpMOHDihfUWplTpH/yd/Vecm1SrA0U4HczzxmzFvnDe7uLhU3vEz9bEu1yhjSPlTL96Rr6e6K14C2IbpjcGv1aB5QwwsCAAAAYHSEdqC6vIKlS++WyoqVIWlHwVG13feW3ItPqJPLESU1/6fSQy6WTCYpP13aNl8qLTxvtxF+Hpp5dVcVlVrO2e5oVqFmr9ivUV0idVXPZpq/8YhWJqRrZcJxrUw4ru7N/TWud7Q6RvpVlOvmrLhgL3tcOQAAAAAHIbQDNeEVbPuywC9K24Pbq9X2WQo4uVktkz6RX+FhHewwUeU17DbCz+O8bc43Iv97cpZ+T86qtG3FPQMI7gAAAEADRmgHLkCZi4/2dH9IkYe+UfT+jxWc9ou8c/Zpb4sblG/nc51rRD4jv0Sr9qZrc3KWyi1/PNf9uy1HNXFAK7k6s+YkAAAA0BAR2oELZTIrJXaMcvw7qvX2F+RemKpOu17QYVOgUiw1HXM/t6pG5OOCvdQzJkCZBSVatCNVP+1MVXGZRbOW7tOnGw7r5n5xuvai5vJy4588AAAA0JAw/AbYSZ5/W23rO0snQ+NltpYrxnxcndbdLfecxHqrIcDTVdde1Fz3XdHu1GcXHcsu0pM/7NbFzyzXi4sTdDy3uN7qAQAAAHBhCO2AHZW7eGtvl/9qX4sJKrOa5ZO1S10WjFJYwjzJaj1/B3bi7uIkSXr3ht6aMaaz4oK9lF1YqleX79clzyzXvZ9v1Z7UnHqrBwAAAEDtENoBezOZdCK4j7ZaWig7qJucygvV4rdH1WHJ+HoddZek5IwCdY7y00vXdNO04e3UNtxHJeUWfb7piK546ReNnr1a769O1IHjZ39GPAAAAADHYoIrUEdK5KJdFz2j8ONr1Pz35+WXtk5dvx+uw13u0LGOt8hqdvmjcc7Raj0eTi4e533uu1T9Z79vOZylLYezJEnX9Y3R6G6RtlH6s+ExcgAAAED9IrQDdclkVmr7G5XZbLBarH9I/sd+UcyWFxR8aIEO9nlKeSHdKwL7V7dWv88xb503uFfn2e+Z+SVae/CkfkvKUEmZRR+tO6SP1h067+l5jBwAAABQfwjtQD0o9onW7sFzFZz4rWI3PiGvzD3qvGis0ltepUOx/1SZJHW5WvIKrbqT/HRp2/zqjcjr/M9+jwv2Uo+YAN1UUqbvtqbo573HlVlQKkkySWoX4au+cYFqGeots8mko1mFmr1iv/KLy6p30QAAAAAuGKEdqC8mk060GK2syEsVs/kZhR74UqEHvlDgoYU6bPJWqmeQ5Hf+W9/tzdPVWeN6N9fVPaO18VCmFu9K1c6UHO0+VvEK93XX5R3CFBvE6DoAAABQ3wjtQD0rcw/SgYufV1rrfynut0flnbFDceZ8he14RokdJionsLND6jKbTbooLlAXxQXqaGahluxO0897jys1p0jz1h2Si5NJknTgeJ46Rfk5pEYAAACgqWH1eMBB8kK6a/vwr3Wg039UanWSZ2GKOm56WK23vSjXohMOrS0qwEM3Xhyr18f30M394hQd6KnS8opH1v3n0y0a9eov+mBNkrIKShxaJwAAANDYEdoBRzI7Kb35SG2xtFRq6GWyyqzgtF/Ubc1kRSZ+KVO5Y0Oxu4uThrQP07NjOuuWS1tIkpzNJu04mqNHv9upi55apkkfb9aKhHSVW+rvOfQAAABAU8Ht8YABlMlJibHXKC1utOL2vCXf7D2K2T9PYUcWKbn1DToZdolD6zOZTLYV4z/4fxdpb1quPt94RLuO5eiH7cf0w/ZjCvN105gezfTPns3UIsTbofUCAAAAjQWhHTCQAt8W2tl7hoKPrVTz/R/Jvei42mx/QbnJ3yspcpTyHF2gpOO5xeodG6jesYE6eDxPS3anaVXCcaXlFOuNlQf0xsoDahvmo8HtQ3V9fMx5V7EHAAAAUDVCO1CXsg/XvI3JpBORA5URdrEiDn2jqKSv5ZOdoM7ZCTph8lFywTEVB7Wqm3rPwd2lYjbN1M+2nLdtQlquEtJy9frKA+oVE6BRXSI0onOEQn3d67hKAAAAoHEhtAN1wdmt4v3nF2p+zCkWJzcdbXGN0qMuV/T+TxSaskzB5lwF/vxvHWt3k452vl3lrr52LPrcIvw8NPPqrioqtZyzXW5RqX7df0K/7KtYTG/joUxtPJSpxxfs0kWxgRrVNVLDO4Ur2NvtnP0AAAAAkExWq7XJrx6Vk5MjPz8/ZWdny9e3/kJQTe3Y8ptGfXpcT3fPUVyzcEeXg/PJPyGVFVevrbOb5BV8ziaex9YrZvvL8jcVSJJK3QJ1pPNkpbW5VlYnYwXgxBP5euDr7Zp7U28dOJ6vH7alaHNylm2/2ST1jAnQ5R3CNKR9GHPgAQAA0ORUN4cy0g7UlfOE8Joq8Gym3Zbm8u8zXjF758oz54DiNk5X5K63daTLHUpvOVYyG+ufdLC3mwa0DdXN/eJ0JLNAC7enasG2FG09kq0NSZnakJSpp3/co+hAD/WODVTfuCC1DfeRk9lUZZ9ebs62RfEAAACAxs5Y/4UP4DxMynKLVPbFryrkyCI12/+J3AqOqeW6aYrcPltHWl2nExH9JbOT5OIh+UY5umCbZgGeuuWyFrrlshY6mlWoZbvTtGRXmtYeOKnDGYU6nHFUX20+Wq2+3p7QSxF+554fT7gHAABAY0BoBxqKP82Tt0pKl3RCQQozmRVlOimP/CNqvfUZRW95UUetQTpu9ZN1zDuGCu6nRfl7aEJ8rCbEx2r9wZO65q11ahPmreSMgkpz5p3NJrUM8Vb7CF+1i/BRablFLy7eq1s+3Fit86y4ZwDBHQAAAA0aoR1oKLyCpUvvrjRP3iLpmKS08iJFpK1UROoKuZflqaUpVdHWE0pJ+FDHu0xUmZu/o6o+Ly+3iv8ZuvHiOEUHeighNVebDmVq06FMpecW21ai1xapZYiXRnQKV+swH4X6uMlkOvtt9EezCjV7xX7lF5fV45UAAAAA9kdoBxqSKubJWyQdDWypY23GK/ToEkUe/FJupVmK3fO2ovfN08nYK5Xa9jrlB3Wu33pryNlsVsdIP3WM9NP1fWN0JLNQmw5lauOhDB04nm97aUeqwn3d1TMmQL1iAtQmzEfmc8yDBwAAABoqQjvQiFic3JXa/Eql+XRWyG/PKtzPQ165BxV64HOFHvhcuUFddbzlGJ2MGaEy96A6r2d/el6t25hMJkUHeio60FOju0cps6BEm089Pm7H0Wyl5hTph+3H9MP2Y/Jxd1aP5gG6KDZQnZv52fsyAAAAAIchtAONkNXsonSrv9L7vSQfS7bCEj5SUPJC+ZzcKp+TWxW3YbqyIvrpRNzfldlskN2f9+7uYpYkTf1sS42PqUqAp6sGtw/T4PZhKiwp17YjWdp4KFO/H85UblGZVu09rlV7j8vT1Ultw3wkSaXl536mPAAAAGB0hHagMTOZlBvaS7mhvXSo8EEFJ36r4MTv5J2xQwEpqxSQskoWk7NyQ3sqK2qgMqMGqtCvlVTFXPHqivDz0Myru1ZaVO5c3F3MivDzqHb/Hq5O6tMiSH1aBKnMYlFCaq42JGXqt8STyiwo1e+HsyRJ172zXsM6hWtk5wj1ax0sN2en2lwOAAAA4DAmq9VqdXQRjlbdh9o72o4tv2nUp8f1dPccxTULd3Q5MLLso9La16TL7pH8os/Y7Z53WMEpKxR0bJU88w9X2lfsGa7c0N7KCe2tnLCLToX4c4+CG4XFatXetFwtPvUouT/zOhX0+7UKVvfm/nJx+uOaeDwcAAAA6lt1cygj7UBj9KfHw51NkaQjko7IW25qqQBTngLCY+R7YovcClLllvS9gpO+lySVufopL7Cj8oM6KT+ws/IDO6jIu3nFs+ANxmwyqV24r/w8XM4I7fkl5Vq+J13L96Sf9VgeDwcAAAAjIrQDjdFZHg9XleL8dKVum6/UXk/K7Bcl7xNb5Jv2m3zSN8jn+GY5l2TLP3WN/FPX2I6xmF1V5BOjQr8WKvRtoSLfivdC3xYqd6vFQnA5R6XSwuq1dfE477Pn/3p7vsVqVfLJAm1PydbOo9nKKfrjUXCuTmaVlFu0fHeaxveNkbuL8f4YAQAAgKaL0A40VlU8Hq5K2YdlkZTjEqKcZiOlZiNlspTKMzdJXtn75JW9T945++SZmyizpUSe2fvkmb3vjG5KXf1U5BmlYq9IFfu3UpF3MxV7N1OxVzOVeEXK4uxe+YCco9JXt9as1jFvVSu4/1nLEG8NbBdqu4V+/cEMrT81B16Snvhht2Yt3ach7UM1onOELmsTQoAHAACAwxHagabuHLfSWyXln3pVcJLUUm4qlYdK5G4qkYeK5WEqkbtK5GYqk0tJtlxKsuWTtUs6uvSMPkvcg0+F+CgVe0epWG4qVp6K21ypYv9Wsji5VV1rfrq0bX71R+XP4vQt9O3CfXV9fIxWJRzXW78cVJCXq07ml+ibLSn6ZkuKvN2cNfhUgO9PgAcAAICDENqBpq4Gt9KfVnzq9Vfm8iJ5FKXLLeeg3A6tkltcvNzKcuWWf1TueUfkVJYv16ITci06IZ8TW/440EnSgdclSaUufir2CFGRR4QKvZqpwLuZCr2aqcgzUvZeNdNsMin21Dz292/qrdJyi37YlqqFO47pWHaRvt2Som+3pMjL1UmD24dpROdwDWgbSoAHAABAvSG0A6j5rfRVsEjKV0vlezWXknZIzf72x+r1VqucS3PlWpgu98JUuRWmVbxykuR2crfcnM1yLi+US2m2XEqz5Z2zv3LfJicVuoepwJSv/F3vKD+0p/L92qjcpYrF46ox9/3PzCaTesYEqmdMoB4a2V6/H87Sj9uPaeH2Y0rJLtJ3W1P03dYUuTmb1aWZn3rHBqpnTIDCfN3P2h8r0gMAAMAeCO0A7K+KW+7LTr0KzjighdTnbjm5usut6LjcCtPkUZAij/wjtpdzWYG8ClPkZZZCkuZLSfNltUqFclOu1V158lCO1VNFcpV06jnz1Zj7fjZms0k9YwLUMyZAD45ory1HsvTjtmP6fmuK0nKLtSEpUxuSMs/bDyvSAwAA4EIR2gHYX01vuXd2k7yCVS6pwMVbBT5xqhSJrVa5Fp+QV26iPE/ulFfeIXnlJ8u9JEOeKpanqVhhypYklbj4KdszRtmZx5WTnajiWoT2PzObTerRPEA9mgfo790ideVrq9W3RZBSswuVnFEgy5/u2XdzNqtVqLeCvd20au9x5RWVXtC5AQAAAEI7gLphp1vuJUkmk0rcQ1TiHqLMkItsm12Ks+SdvVfe2Qnyyd4jn+y9ci3NVkj2NoWYJa2coCLvaOWE9VV2eLyywy9WqWfoWU+xPz3vvGUcOF6xJN/fukYqLthLecVl2n4kS78fztLWw1nKKSrTzpQcW/vr3/1Nl7QO1kWxgerRPEDtInzk4mS+sO8FAAAAmhRCO/D/27v74Krqe9/j7/WwH5LsJDsEEgyNRAJIz7EQlIfiFBFLxRkfJvfaORxrBZHxzLRXpzRyqlYFH+5cbIUO12Kx9Vbs2HFAO+r0qIOnRrnaS861pc0RRRkvKiASCE952DvZD2ut+8fe2UlIgCRAspN8XjPbtdZv/dba35X8gvv7+6312zJsJQJhTpTM4URJKpE3nRihpj0UHtpBwcH/TciME2w9QLD1ACV7XwIgkj+Jk+Nmc3LcbFqK/oFgNHUr/cqt9X1+36AvlXiHAjbzKscyr3Isrufx+dEI/3ngJDv3n+Czxggn2xK8/sEhXv/gUOa46V8LM/PiMJeVFXLZhEImjsnFNI3z+FMRERERkZFESbuIjBiuFaB5zHSaA2VwYB+m61BAGwVGhEIjSh7t5LV8Rl7LZ0z4bCtJz6SJPF72lbB/3n8nWnzZWd8j6DN7fAc8pCayqxwXonJciJkXF/HTV3bx+H/9BkdaYuzcd4K/7z9Bc3uS9z8/zvufH88cFwrYfP2ifCaXhJg0NkRlSR6Txob4WlEOtkblRUREREY9Je0iMvKkn6l3kzFOAifTxXaihXDTx4SbdhNu+hhfspViWii2Wrj8/VuIFk7h5IQFnChbQEvJLLwzfWd8HwR9FtdMK+GaaSW4nsfBE2183NDMp4db2dvYyudHI7TGkr1ObGebBmXhHCaEc/haUQ6lBUGKQ37Kx+RyxcVFhHN9GIZG6EVERERGOiXtIjIy9fJMfRI4OnYaR/kv4LnkNX9G+OB2wl/Wkm/EyG36lNymTynb/b9w7NzUc/AXfYvm0rlEw1PB6NvId8ft8/255b5HrK7H/uNR9h/vOdc+pCa9G18YpLQgyEXpZTjXRzjHT1Guj8KO9bzUMugzleSLiIiIDENK2kVkdDJMIoWTiZDDwf17sRc/RmHsEOGvthP+6l38bY2M+bKWMV/WApAIjKG5dA7NJXNoHVtFZMzXTzsSf1FhDr/4pxm0J9wBheZ6Hk3RBI2tMRpbYhxtjXEymuBYJMbR1jgAsaTLvmNR9h3rPak/ld82KQja5Pptcv0WoYBNbsAmz2/heWBbBkGfRa7PIui3CPpMcn02AZ9J0GeR47MI2CZj8vxcOj6fXL+N39bt+yIiIiIXmpJ2EREg2dbEscJLOXbppTD1X8ht+Yxw418oPPaf5J/4EF/sOMX7t1G8fxsArukjWvR1WsfOIBqeSrRwCm3hqSQDYYBen3s/V58fjfDTV3axblGYcNDkaNThWJvLsTaX420urXGXlrhHSyy9jLu0xj2SLsSTbjrhj5+3eCzTIGibBNJJfdBnErDTCb7PJBSwKSkIkOe3yfFb5PotcvypjoKu66l9qc6E3PS6dZrJ+T4/GiESS/YpvryAzSVj887b9YqIiIgMBSXtIjK62enR8nfXdSuOpl9fAQaXkEcbhUaUfCNKKGDjizcROvYBoWMfdDsunlNCe6iceN5FxPLKiOWVkQiOJekvxPEXppa+PFzLj2f68Exf6rZ718F0ExhuHMNNYLjJzm0ngenGKTu8jyvNBl5/O0GABH4SBIzUejEJik+5tAQWMXy0GX5ayaGZXJq9PFrIJUKQKEGSWIDBN8tzMP25xJMusaSbXjrEHZd40iPhuLQnHGLJzrsHHNcjEneIxJ3z/VvBlx75D/osgnZqtN80DPYcbunXef7nP1dxydg8zDM8GqDkXkRERLKZknYRGd3Sk9aRjJ22ige0Aq2RI/DBizDrHgJ+H6GTnxBq+pSc1n3ktuwj0H4Ef1vqRWPfQ/AwMPD6VPdqf9/P2xcOFgnPxGstJhkIk/QVkggWkPQXkvCnl77O7YNuMSdyK4g7RjqhdzuX6fVE0uV4NM7bnxwZcFwJxyPhJGlp79uo+un8aEt9n+r9t4WVVBTnUZDjoyDooyDHpiDoIxSwCQVtfOmZ/C/ESP+FuntAdyWIiIiMDEraRUR6mbSuVx2j8u+tJwbEgGOZncVYhAkSJ9BlBNxvJPGRxMbFxsHGwTS6J+i9JeyuYadG4i0fnuHDtfy4GHiRY7ihUlxfHq7pwzP9qaVh42VGkw3Aw3STGG4C00tguAmsZBt2MoKdiGAloxi4WDhYhgNtDanXWcwk1cmQDIRJBopIBIpIBsaQCBaRDBSRDIRxgjm44Tz+JdRKW9IkaQVJmkGcjqXpw8PEMyw8w8T15eKFSnGxSHqctiMgHjmJ+9Er5BCnHT8x/LTjI+b5aCNAlABRgkS9IBGCtJKDg9WnX+1T7+w94/6gzyTHZ3EimujT+Tr8/LvTuWRsHkHbIseffnzA33kHwYETbSxct71f53xn1dVnTbA/Pxq5IOcVERGRwTdikvannnqKJ554goaGBmbMmMEvf/lL5syZM9RhichIcpZReQeIpF9n5LkYnoPpJTFaD2PsfhVv3l244Qo808YzrN5nqm86kLqN/x/vgsIJ53YtnovltGMlItgtX2LFmrCdKL5kBDvRii/Zmkrwk634Eh3LFmy3HQMPX+wEvtgJzvTk/uSBhIWBZ9qAgWeYYJipJakOCc8XBTuAZ1qAiYcBhoFnWLjpDgzH9ON6Hm7LYdrGTafZV0IzIZoI0Ux++jGBHFq8HA61+/i/h2FGxXg8y0dzHJpjLs1tCZrbE5nJBNsT7oAmFvzJHz444347/ex+0Gfit0ws08AyTWzTwLaM9LaBbRokHI/Pj0Z49N8+orQgiN828Vkmfjt1bNdlY0uqjV77D6WMyw+kz2FmztWxtK1U3d/VfUFjSztl4SB+6zx908CxvRDr4+MMgXworjxrNd09ICIio9GISNq3bt1KTU0NTz/9NHPnzmXDhg0sXryYPXv2UFJSMtThichI0tdR+bPwSCX5BMKADXVP9/1g+9y+Px4Aw8Sxc3HsXOI54/p2TNNBjLpfYl/7CHZuIb7YCez249jpBN5uP44vfhIzGcWKHsNs/AgzpxALF9OJY7kxTDeO6Z3+GXgDD8M9w2i2ATjR9A/vbNcIHP0/Z68XBLrcZJAa/Q/hFISI2/m0+MbQYoZpcPL5jwYoG1+GEyikxcil1cuh1QsScf20uj6ijkXcNWiOuew70U5ZUR5g0pb0aE+6tCVSyw5JN3WXRX86Bd7Z0/dnL/599+E+1/2nX/9HZt1nGfgtE1/XDoFTOgd8vZSn5g7wMGLN8MlrmbtIOpfd17v5+k0QDHe5Y6ST50FLLMEbu85+N0hX1102nvyAjWGQ6QzJdFxYBpaRWm9uT+K6HqbZ2VFiGqnJFi0jvZ1ezw3YXFQYzHR62JkOkO4dLvap61Znh4nP6uxA0VcxiojI2Rie5/XtQcosNnfuXGbPns3GjRsBcF2X8vJy7r77bu67776zHt/c3ExhYSFNTU0UFBRc6HAH7MP697lhSyP/Y2Yzl3xt/FCHIyLnS+ToGZ+p78YOnLeOg35rOgh1G+GqVVBYfpa66bsC5vVyV4DnAS6G52Kc/BLe34Rx5V0Y+RdheA6G65Dq1vAwPBc8L5XoNX+F8dffwoxbMELjUnU8DwM3NXGfk+oUsNJLs+0YZqItVebGMZ1Y6u4Ctx3LiWE5bXjxKJG4Qx5thGjv8ejCheB5pCYIJEAbfuKejzh2+uUjho8Evsx6HD8J7NRjAG7qjoKkkaofw0fCSx3XcXwCm3bPJurYOKaPJFZqv2eTwCLu2Zn3S6TXk318jEDOP8vwsA0w00vL9LAMUusG2B3bZmo7lfQbqQ6QdL5v0NkJYhjdO0S6bnc9BtJ/ikDccXHdzrbvpR+xIfPf7vVP5XV5x44qpgG2afYo7+2cScfF9TjtzB5d39cwUh0hp5b3dv6k49HxMbfj52Bk1r3MumlAwDIxjM4rMdPrRvpaOjqbEk7nv0md5/R6nBfAcuMELbfzfTvqZY5Nb5smhu3vXs/wMDwyMXW8Yo6L56aP7/ZIFJlfrkeqM8gyDXL8dpd9qXbTsU6X41OrXcu6nbmn0+w4Xf3TlvfzPK4HLqnfvev13PbSdTzPIJJwSDpe7/sB1zPoaPaZOnT+zk0620fHtm0a5PrNTJmV/pGZdFnvuo8ux2e2vcwxhtHzvTrbXHp5SnlH3Y5zdNTp+jPr8bd0mr+V1HaXv1/v1H1nOO4M5zz1/bx0/W7rp26fun7qko4O3CQJx0u9xSn7oPN3aZkmN82ZwuWX/eOp0WWVvuahw36kPR6Ps3PnTu6///5MmWmaLFq0iLq6ul6PicVixGKdH5CbmpqA1A8tm7W2tuLGouz9qpFotG/fzSwiI00MGKJ/q2LN0F4M/765jwcUw/5GCJyhQyLWDO0l8PaLfTxnGRz1QUvXUWkT8KdfXVWkP9Gc4XSxZo78v5204Qc8fCQJkMCHQ8BIzdDvJ5l6GUnyaCdstKbnKXDwpecp8BlOOvV1MfDwMIhjY6fnDeiY08DKdArEsYgT6uNVd9Ofu/TPVjf9Wc31jHREdrrDIJXQJ7CJpTsDktjEPSvTMRDHznQAJNN13G4pS+fnNu+UN+ws730Sxt4/sPetXm9lLuBi4ngWDiZJOpcuJk76lcTC8Uyc1GwPJDFxMTL1nfT+JFa6zEjXS788K32sSZKu9SySp/nI5QL9mylBso9xmvXgYAdyFj3/IkVGLodQ7D0mX3yWQYYh1pF/nm0cfdgn7UePHsVxHEpLS7uVl5aW8sknn/R6zNq1a3nkkUd6lJeXZ/cvtcPqoQ5ARKTP3hgm5xQREZGR5F+Bf/3hD4Y6jD5paWmhsLDwtPuHfdI+EPfffz81NTWZbdd1OX78OMXFxVn9bFlzczPl5eUcOHAgq2/jFzlXausymqi9y2ihti6jhdq69JXnebS0tFBWVnbGesM+aR87diyWZXH4cPfJdg4fPsz48b0/9x0IBAgEuk/kFA6HL1SI511BQYH+AZBRQW1dRhO1dxkt1NZltFBbl7440wh7hzM96Tcs+P1+rrjiCmprazNlrutSW1vLvHnzhjAyERERERERkXMz7EfaAWpqali2bBmzZs1izpw5bNiwgUgkwvLly4c6NBEREREREZEBGxFJ+5IlS2hsbGT16tU0NDRQVVXFtm3bekxON9wFAgHWrFnT49Z+kZFGbV1GE7V3GS3U1mW0UFuX821EfE+7iIiIiIiIyEg07J9pFxERERERERmplLSLiIiIiIiIZCkl7SIiIiIiIiJZSkm7iIiIiIiISJZS0p5lnnrqKSoqKggGg8ydO5f333//jPVfeuklpk2bRjAY5Bvf+AZvvPHGIEUqcm7609Y/+ugjbr75ZioqKjAMgw0bNgxeoCLnQX/a+zPPPMP8+fMpKiqiqKiIRYsWnfX/BSLZoj9t/eWXX2bWrFmEw2Hy8vKoqqri+eefH8RoRQauv5/ZO2zZsgXDMKiurr6wAcqIoqQ9i2zdupWamhrWrFnD3/72N2bMmMHixYs5cuRIr/V37NjBLbfcwooVK/j73/9OdXU11dXVfPjhh4McuUj/9LetR6NRJk2axOOPP8748eMHOVqRc9Pf9r59+3ZuueUW3nnnHerq6igvL+faa6/l4MGDgxy5SP/0t62PGTOGBx54gLq6Oj744AOWL1/O8uXLefPNNwc5cpH+6W9b7/DFF1+watUq5s+fP0iRykihr3zLInPnzmX27Nls3LgRANd1KS8v5+677+a+++7rUX/JkiVEIhFee+21TNk3v/lNqqqqePrppwctbpH+6m9b76qiooKVK1eycuXKQYhU5NydS3sHcByHoqIiNm7cyNKlSy90uCIDdq5tHeDyyy/n+uuv57HHHruQoYqck4G0dcdxuOqqq7jjjjt47733OHnyJK+++uogRi3DmUbas0Q8Hmfnzp0sWrQoU2aaJosWLaKurq7XY+rq6rrVB1i8ePFp64tkg4G0dZHh6ny092g0SiKRYMyYMRcqTJFzdq5t3fM8amtr2bNnD1ddddWFDFXknAy0rT/66KOUlJSwYsWKwQhTRhh7qAOQlKNHj+I4DqWlpd3KS0tL+eSTT3o9pqGhodf6DQ0NFyxOkXM1kLYuMlydj/Z+7733UlZW1qOTViSbDLStNzU1MWHCBGKxGJZl8atf/YrvfOc7FzpckQEbSFv/85//zG9/+1vq6+sHIUIZiZS0i4iIZKnHH3+cLVu2sH37doLB4FCHI3Le5efnU19fT2trK7W1tdTU1DBp0iSuvvrqoQ5N5LxoaWnhtttu45lnnmHs2LFDHY4MU0ras8TYsWOxLIvDhw93Kz98+PBpJ94aP358v+qLZIOBtHWR4epc2vu6det4/PHHeeutt5g+ffqFDFPknA20rZumyeTJkwGoqqri448/Zu3atUraJWv1t63v3buXL774ghtvvDFT5rouALZts2fPHiorKy9s0DLs6Zn2LOH3+7niiiuora3NlLmuS21tLfPmzev1mHnz5nWrD/CnP/3ptPVFssFA2rrIcDXQ9v7zn/+cxx57jG3btjFr1qzBCFXknJyvf9td1yUWi12IEEXOi/629WnTprFr1y7q6+szr5tuuomFCxdSX19PeXn5YIYvw5RG2rNITU0Ny5YtY9asWcyZM4cNGzYQiURYvnw5AEuXLmXChAmsXbsWgB/96EcsWLCA9evXc/3117Nlyxb++te/8pvf/GYoL0PkrPrb1uPxOLt3786sHzx4kPr6ekKhUGaERiRb9be9/+xnP2P16tW88MILVFRUZOYpCYVChEKhIbsOkbPpb1tfu3Yts2bNorKyklgsxhtvvMHzzz/Ppk2bhvIyRM6qP209GAxy2WWXdTs+HA4D9CgXOR0l7VlkyZIlNDY2snr1ahoaGqiqqmLbtm2ZiS7279+PaXbeHHHllVfywgsv8OCDD/LTn/6UKVOm8Oqrr+ofAMl6/W3rX331FTNnzsxsr1u3jnXr1rFgwQK2b98+2OGL9Et/2/umTZuIx+N897vf7XaeNWvW8PDDDw9m6CL90t+2HolE+OEPf8iXX35JTk4O06ZN4/e//z1LliwZqksQ6ZP+tnWRc6XvaRcRERERERHJUuoCEhEREREREclSStpFREREREREspSSdhEREREREZEspaRdREREREREJEspaRcRERERERHJUkraRURERERERLKUknYRERERERGRLKWkXURERERERCRLKWkXERGR8+rhhx+mqqpqqMMQEREZEZS0i4iIjHC33347hmFgGAZ+v5/Jkyfz6KOPkkwmu9Vbv349RUVFtLe39zhHNBqloKCAJ598crDCFhEREZS0i4iIjArXXXcdhw4d4tNPP+Wee+7h4Ycf5oknnuhW57bbbiMSifDyyy/3OP4Pf/gD8Xic73//+4MVsoiIiKCkXUREZFQIBAKMHz+eiRMn8oMf/IBFixbxxz/+sVudkpISbrzxRp599tkexz/77LNUV1czZswY7r33XqZOnUpubi6TJk3ioYceIpFInPa9r776alauXNmtrLq6mttvvz2zHYvFWLVqFRMmTCAvL4+5c+eyffv2c7lkERGREcEe6gBERERk8OXk5HDs2LEe5StWrOCGG25g3759TJw4EYDPPvuMd999lzfffBOA/Px8nnvuOcrKyti1axd33nkn+fn5/OQnPxlwPHfddRe7d+9my5YtlJWV8corr3Ddddexa9cupkyZMuDzioiIDHcaaRcRERlFPM/jrbfe4s033+Saa67psX/x4sWUlZWxefPmTNlzzz1HeXk53/72twF48MEHufLKK6moqODGG29k1apVvPjiiwOOaf/+/WzevJmXXnqJ+fPnU1lZyapVq/jWt77VLQ4REZHRSCPtIiIio8Brr71GKBQikUjgui7f+973uOGGGwiFQpk6v/71r7n11ltZtmwZzz33HGvWrMHzPH73u9+xfPlyTDPV179161aefPJJ9u7dS2trK8lkkoKCggHHtmvXLhzHYerUqd3KY7EYxcXFAz6viIjISKCkXUREZBRYuHAhmzZtwu/3U1ZWhm3btLW1UV9fn6lTWloKwB133MHatWt5++23cV2XAwcOsHz5cgDq6uq49dZbeeSRR1i8eDGFhYVs2bKF9evXn/a9TdPE87xuZV2fgW9tbcWyLHbu3IllWd3qde1UEBERGY2UtIuIiIwCeXl5TJ48uVtZTk5OjzKAyspKFixYwLPPPovneSxatCjzfPuOHTuYOHEiDzzwQKb+vn37zvje48aN49ChQ5ltx3H48MMPWbhwIQAzZ87EcRyOHDnC/PnzB3yNIiIiI5GSdhEREelhxYoV3HnnnUDqmfYOU6ZMYf/+/WzZsoXZs2fz+uuv88orr5zxXNdccw01NTW8/vrrVFZW8otf/IKTJ09m9k+dOpVbb72VpUuXsn79embOnEljYyO1tbVMnz6d66+//kJcooiIyLCgiehERESkh5tvvplAIEBubi7V1dWZ8ptuuokf//jH3HXXXVRVVbFjxw4eeuihM57rjjvuYNmyZSxdupQFCxYwadKkzCh7h82bN7N06VLuueceLr30Uqqrq/nLX/7CxRdffCEuT0REZNgwvFMfMhMRERERERGRrKCRdhEREREREZEspaRdREREREREJEspaRcRERERERHJUkraRURERERERLKUknYRERERERGRLKWkXURERERERCRLKWkXERERERERyVJK2kVERERERESylJJ2ERERERERkSylpF1EREREREQkSylpFxEREREREclS/x9IAbx0GnJwHgAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAABHIAAAPaCAYAAAD7o3/hAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOyddXhT5x/FT5I2qbvT4u7u7u7uMnTO3N1+bINtjAHb2Ia7uwx392IFCnX3tI38/jjNYjfF2tLC+3mePaO5Nzc3ae+b+573fM9Xptfr9RAIBAKBQCAQCAQCgUAgEBR75E/7BAQCgUAgEAgEAoFAIBAIBA+HEHIEAoFAIBAIBAKBQCAQCEoIQsgRCAQCgUAgEAgEAoFAICghCCFHIBAIBAKBQCAQCAQCgaCEIIQcgUAgEAgEAoFAIBAIBIISghByBAKBQCAQCAQCgUAgEAhKCELIEQgEAoFAIBAIBAKBQCAoIQghRyAQCAQCgUAgEAgEAoGghCCEnEdEr9cjNTUVer3+aZ+KQPBcI65FgaB4IK5FgaB4IK5FgUAgeH4QQs4jkpaWBnd3d6SlpT3tUxEInmvEtSgQFA/EtSgQFA/EtSgQCATPD0LIEQgEAoFAIBAIBAKBQCAoIQghRyAQCAQCgUAgEAgEAoGghCCEHIFAIBAIBAKBQCAQCASCEoIQcgQCgUAgEAgEAoFAIBAISgh2T/sEBAKBQCCATgtkxAI6HeDoCSidHu84aTE8lsoFcHAr2HMUFB7psYBWAygd+fsXCB5EZiKgUQMKJeDs87TP5tlFqwFS7wN6HeDgATh5Pe0zEggEAgGEkCMQCASCp01qJHBuKXDyDyA3E6jUBWjzDuBVDpArHu4Y6bFA6Gbg8M9AZjxQuinQ/mPApzJg71C45y94fDLigbB9wIH/ASkRQGAdoMMngH8NinECgSXqVCD6IrD7UyD2CuBRGmj7HlC2BeDk/bTP7tkiKRy4uBI4/Rc/93Kt+Fl7VxLjqkAgEDxlZHq9Xv+0T6IkkZqaCnd3d6SkpMDNTaz2CgRPC3EtPiOkRgHLhgJR58wfV7oAk/YBPpUefIyMBGDLdODKevPH5Qpg3DYgpEkBnaxAise+FtWpwP5vgaO/Wm8bugyo0g2QyQruRAUlH50WuLweWDPeelubd4DmrzzXAmCBfi8mhwNrJwPhR8wft1MB43cCQXWf7PgCgUAgeCJERo5AIBAInh7RF6xFHADISQf2fQfkZDz4GGmR1iIOwEnfljfo+hAUPzLigGNzpLdtfQNIiyra8xEUf9KigG1vSm87+D3/pgQFQ8JNaxEHADTZwO5PxLgqEAgETxkh5AgEAoHg4dFpC+5Yej1wYaXxZ5kMKNsSqDWIpVHXtwHqlAcf584h29uiLz7cMQRFT+xV/g1IkRoJqJOL9HQEJYCsJGbjSKHTAkl3rMeoghyznidCt/L/dg5A5S5ArYGAb1U+dns/kJ369M5NIBAIBCIjR1AyiUjOwlurzkMmA74bUBvBno8ZjCoQCB5MrhpIuQecXw7EhVJsqdIdcA8B5E+wHiCTGQOJy7cFmr/MvJTE20CZFiyVkD1ERo4yn1IKmfzhc3YERYu9Y/7b5eIWRWDBg/4m9Dpg9QRmuZRuClxayzGrTAugao8nH7OeJ5QuQMPxQKVOwLXtQGYCUG8k4FkW2PUxAFH2KBAIBE8TcZckKHHodHpM+PskEjJyoNfrMWnhaWx+uSXkcnFTIRAUOJpcrr4uH2Zc2Q7dDOz9Ghi3FQio9WTHrzcKiDwL1B8NLBsGaHOMr3HsN2DMJsDVP/9jlGlOwUavs95WpZsIQC2ueFeimJObZb0tqB7gKLrjCCxw8mKAefx1620O7nSJpEfz72peK/Mxa983BTNmPS/UGQJc3chx2UDoZsA9GBj4F+Di9/TOTSAQCASitEpQ8th0IRKh0Wl4uV1FvNiuIq5EpWLf9dinfVoCwbNJehSwerx1eUJ2KrBu8pNnUniWATp/AWx90yjiGMjNBNZMANKi8z+Gqz/Qc6bE44FA5y8BleuTnaOgcHANAAb8SRHOFAcPoM8c0VJaYI2LP/9mLK9puR3Q7TtmLjWdCmx7W3rMWjuJHe4ED8fer60fS7kPHJtLkV8gEAgETw3hyBGUOBYfu4tapdxRyd8Ver0eFXydsfDoXbSv+oBVe4FA8Ogk3WXwsBQxl5lX4ez7+Md38gbsHG3nXiTdYaima4DtYyhdgJoDgODGwNnFLAOr0g0o2wrwCHn8cxMULnZKoEJ74MXjLNuLvw6UawNU6syW0gKBFP41gSmHgGtbgagLdHYF1gKOzwOiL1EQzk6zfp5cQeE5K0m4SR6GGzuN/3YP4VgacRbQZAFXNwDt3wecPJ7a6QkEAsHzjhByBCWKe4mZOHknCdPaVgAAyGQyNK/gg+Unw5GRrYGzSvxJCwRm6HTs6pQcTrHEpxLg7Ac4eT7c8zXqBxzfYlU2MxHIjAdiQzlx8qnCSZODjVa46hR2Qcn3HLI4MTOswudmcVU9/gYAHeBdma/hXx3o+nXeSrzM+tz+e81UICMWiLsOKJ2Z+eDiD9g75H8egoLH3pGlMh0+5u9N5Bk9f2QmAGkxQGIYRWGPEMAtyHwfvZ4CjZ2KGTfOfszp8qrIkkq9jn9LjScB9haZeQol0OoNtstOvsfSK7kd4BII5KQAKRF0/XmU5jjg7GP+d6jJ4XMSb9Ml6FOZ52lrTHtWyE4HGr4ANJnEbmHpcUCXyty2ZCC/WwQCgUDw1BCzXkGJYteVGNgrZGhU1pidUL+0JxYdu4vDN+PRuUY+q/YCwfOGTgtEngGWDOIqtIEa/YCu3+bvcjHgVcF2/oyLH+BoIgilRgHnlgD7vjaWNSiUfK1qvQEXC+dOehyw/1ugYgfuZ1laBQAqN060rm0Hmk3jBOzSWmDbW4A2T6iR2wFdvgJqDwMU9nTknFkExF9jyGn1vpwcyhV8zYMzgBPzjR2TlM5Ar1+4j0eZB2fyCAoHIeI8f6RGAuunAWF7jY+5BgAj1gD+NSgkJ4fTaRcXygDjGgOAGzuAHe8DOg2fo7AHOn0JKJ04HhjGLLkC6P87cHYhM3IMOPsCw5YB+74Fbu42Ph5UH+j2LSCzA7zKURS6fQBYPRbIyeA+MhnQZBrQ6vUncyMWd2oPBbISgb+7m7car9gJGL9DhJELBALBU0Zk5AhKFPuvx6FqgBsc7I03/AHuDghwc8Dhm/H5PFMgeA5JiQAW9jUXcQDg8jrg1AKjEJIfLr5Ai9elt3X/nqvaBmKvAP9+YZ5Noc0BtkwHku9aP//6NuDkH2xB3vwV6ddo8zZw5h/g4PfA/dNAwk1g82vm567TANveAeKuAncPAXOaAkd/YWnA7k+AuS2AmEvc9+YulmCYtr3OyQDWTaI7aN2UB2fyCASCJyc3i0KKqYgD8Ppb1Jf/v30A+K0ZcORnXs9HfgGizudl4GiMz9HmAtvfobB8YzedOQBQrQ874d3cY/4aGXEUuOuPNn888gyw/zvgynrg4EyKSMuHGkUcgGPHsV+BW/sK5nMotujpvMmwuLe6uYtB9DkZdDcKBAKB4KkghBxBiSFbo8XxsATUKuVuta2SvwtO302SeJZA8BwTccp2vs3xuUB6zIOPoXIFmr0IDFoI+FXnz6WbcUW2QntjK9/0OODwT7aPc2yu+WQoLRo48D3/fXkdj9N7NhBYh69RqgEw6G8Ga976N+/9nAYO/2z7NQ7+CNz819o9lJPOkNPkcODADOnn6jTAnYMs47p7JN+PRCAQFADpMcD5ZdLbDFk2ayaYC8M1+gPnFts+5vllvIZdA4EePwC1BwMXlkvvm5WUl5dj4cC79S9QphkQdRa4tMo6NNnAge+e7eDkmMuAOll627nFHLOfNOxeIBAIBI+N8EUKSgwX76dArdGhRpB1XXplf1f8ffgOMnM0cFKKP2uBAAAzJ2yhTpEuZZLC2Ruo0Qco25zPsXcyL6kCmB2Rcs/2MVLuUuzRx7D8ATIg9b5x+/7/AV7lgTrDOAmzU3FCdWYhSy0aTmAJ1o0d+bxGOFCqnvS2uFCuHqfcl94OcJuzL0WuCu0BRw/b+woEgidDk217DLJTMZfFMrTY2QcIP2r7mCn3mcu1+xPmgfWbJ93e3kBqBAPXTUVtvR5w9GIGj8oVGLqEYk5WEnDyTyD6gvG1HsbVWFJJum17W24mf3+yh/wOEQgEAkGBIxw5ghLDqbtJcLCXo4y3s9W2Sn4u0Or1uHA/5SmcmUBQTAmqb3ubezC7RT0Kzr6AWylrEQcAHNyBgNq2nxtYjyvj6yaxBCv6ItD2ffN9EsOAvV8BG1+iY+f+SR6z18/A9e10/OT3GkH18xev9DogoKbt7QG1+HxNtnQmkEAgKDjsndhqXgq5nfQ1mHjr4a5hgGHo6bH5t7H3qcycHlMUSoa2Z6dSzF0xClg5Gjj4A4N/6w7Pe63aDFh+VgmsY3ubix/fu+oZD3wWCASCYowQcgQlhtN3k1DR1wUKucxqW7CnE+wVMlyJFPXaAsF/+FW13ca5/ceAW6D0tsfB0QNo+bp0YK2dA7MoDv5AcWbVWAB6wLcqV80tcfIGvCvQRdPxE5ZXRJwGru8AGr3AiZYlcjug6TQgdIv0+bkGcELX8XMb5+8J+FZhlk7twbYnmAKBoGBwDWQ3KSk8y7ObnGWg7tVNQO0hDDe2RKFk6dU1kzHg6K9A67ekX8OnEsOULcuH6o5ge+193wB3DgE1B/Dx5HBgw0tAxY4Uwjt+Cjh5WR712cGjNMdhKVq8Djh4WpelCQQCgaDIEEKOoESg1+txJjwJFf1cJbcr5DKEeDrhapQQcgSC/3ALAkZvBMq2Mj7m4MGQ4kqdCv71vMoDw1ey85MBn0rAyDV54cp5Nnx1CssjrqwH+s4z7/wSVB8YvYFdasq1AW7tNZZXVGgP3N4PDPjDfILhWY6daVLuA81elj63Hj9y4hhYh9k7Ln4mr1mPz9/1CeAeAlTvY8z+EQgEhYPCju6WDh8DShc+JpMBlbuyo5RbkLVrT5MNHJ3DccarvPFx7woc6yAzF2FzM4BKXSjgmrpHyrdj7lf4caP4bOdAoTi4IXBhBR87/RfHA1OOzOZz/WsVxKdQfFEogWErgPJtjY+p3ID2HwGVuwGZcfx9CQQCgeCpIMJEBCWCmNRsJKTnoLyvdVmVgdJeTrgihByBwByvcsCQRUBmApCrBhzd2WlKUUDDf04GhRmdjhOhcu2AMRuZJ6HTUrzZ9y0FGFMizgCeZeiSmbSfq+IKe8DRm5k8vX8GstOBDS8anxPcCLi0hsJO48mc6AHMt9j/LQWd7jOAyl3o3jm3GPCqCHT4EPCtxkmHNgcIaQpMOsDMntx0IPIcsOcLoGpPoOF4rrYLBILCx9mH4mutQSxlUuU1M7BTsfSq4TiWUu37lp3v/GsAbd/l/8dtM3bkc/QCXP05DpmOJ07e/K/ZNKB6LyArGdDlctw6/RdFnxf2sLzK2Q/Q5QCaHKDzV8CpP4GEWyzxcvahE8g9mPu6BQEO0gtLzwwp9znW9pnDLCBtTl4pmRxYNQpo8y5FeDvV0z5TgUAgeC4p8ULOr7/+ihkzZiA6Ohp16tTBL7/8gsaNG9vcf9asWfjtt98QHh4OHx8fDBw4EN988w0cHByK8KwFj8rFCGbflPOxLeSU8XbC4VvxyNXqYK8Qq+kCwX84ekrn2jwJmmxmUeyfAdw7Sot9/TGAVwVa8h19eLMfeVb6+a4BQG42SyfcS/E/U5x9OalzD+HKe8+ZfMy/JnBiLlsNG1qIy+RAt++4grxiJDuplGnJVXvXIMDJE0iLoQPo6K8Unip1Bpq/zHIqr/JAjX6crEmVbAgEgsLDTskxI+kOcHwey5oUSrpjqvelMFuqIaDNzgta9+DzlM4cR0yRy6XHE4U9XT/3jnMMyExg9716IzmO2DsBh2dyvHILAhq+APSdy/FJbg/0mAmc/AO4tBbwrsisHnsnCuPPKkpnimDx14FDM/l/70pA6zeBLl/TySREHIFAIHhqlGghZ8WKFZg+fTrmzp2LJk2aYNasWejSpQuuXbsGPz8/q/2XLl2Kd999FwsWLEDz5s1x/fp1jB07FjKZDD/++ONTeAeCh+VSRArcHOzg7SyRjZFHsKcTcrV63EvMRHlflyI8O4HgOST6AvBXN2PXltRIToLqDqeYE1iXwo4tIad6byD6cv4ZC3ZKoOVrQNx1YPu7QOxVClL1RgJ1RwLrJnOVuOV0TtAurjY+9+JKCjfjd3JFfdPLdOkYOP0XcHEVMPFfijkCgeDpkXALWNDFvJ319ndZ4jR02ZPneWUkADve5zVv4NIavm77j4Clg4zhyqmRwP1TQNOp7IDl6MmxKOI0kJPOblp/HwR6/QLUGcptzyJO3gyZ3zLd+FhaFHDnAIX1il0BhRByBAKB4GlRom0LP/74IyZOnIhx48ahevXqmDt3LpycnLBgwQLJ/Y8cOYIWLVpg+PDhKFu2LDp37oxhw4bhxIkTRXzmgkflcmQKyno7Q5ZPPXaQB7tH3IxNL6rTEgieT9LjgM2vS7fePbcUCKwNRJziim6Dsebb5Qqg42dAQhhQf9SDS7wSbgKrRlPEAVhKceQX4OwillgolMy0MBVxDGhzgK3TgYwYcxHHQE46sOcz6xbHAoGg6MhIAo7ONhdxDESeBSLPPPlrpEaYizgGGowBtr4h3SHr+Dw6e+wdgAPfA52/NN++413ztuXPGuoUYPen0tt2fghoM4H06CI9JYFAIBAYKbFCTk5ODk6fPo2OHTv+95hcLkfHjh1x9OhRyec0b94cp0+f/k+4CQsLw9atW9G9e/ciOWfB43MtOg0hXk757uPpZA8npQK34jKK6KwEgueU7BS2D7dF1AUg5jK7Ttk5AGM2AX1+BfrNAybuBap0B+oMsy6LsCQ1Ctj+nvS2W/+yNKLvPJZ4ObgDFTsAFToYg1MBZvFkJtl+jWvbOGERCARPh8w44NpWho5X7mLtkDu7iLk1T8LNPdKPO/mwpEsKvY7bEm5R7FW5mJeo5mQ820JOWjRzixT2DMyv1NnooMzJoKCf39gqEAgEgkKlxJZWxcfHQ6vVwt/f3Jbv7++P0NBQyecMHz4c8fHxaNmyJfR6PTQaDaZMmYL3339fcn8AyM7ORnZ29n8/p6aKMN2iJiNbg3tJWehRO39rtUwmQ5CHI27FCUfOs4i4FiXISGAA6MXVgF4L1OzPvBfTLlCFgewBawAKJZ03Og1wfC5wYj4w5TAnQfdOAHf2s/V4xU7MwJFy5WhymGORcs/268RcAZz8gIBaQM9ZDFSWybnKHneN7YOhB2QSLdH/O1d7AKLzyqMgrkVBgaLNAfr+xvLI5HCKBl7lgN2fMZfFzoHB6Qm3gNDNFG7LtQGCGwMeDwgmz80C0uOlHTfAg7suKVSAHhxf758E/KoBd48Yt1u2Ry9iCvValCuAhhOAyp2BsH0Uszp8DEAPbH+f2+0dC+71BAKBQPBIlFgh53HYt28fvv76a8yZMwdNmjTBzZs38eqrr+KLL77ARx99JPmcb775Bp999lkRn6nAlOsxLHsI8czfkQMAQe4OuBEryiSeRcS1aEFGHCc6ZxcZHzs+F6jaC+j5Q/7ZM0+KoydQtiVw55D1Npkc8KvKm/wzC/mYWylO1uY0ZTcZA3YObDUe3Ni83bdWw0lTVhKPZ3MSBtr7j6/gBM/AqQVAzQFA5y/YvtzZ2/Z7qTUUcPJ62HcugLgWBQWIVgNkxgPLhjJA3YCTF9D/d2Djy0D9sRR5lvSnoAMAp/8GXAOBsVvYelyKXDVwYxewejwwdIn0PolhDFCPuWS9TaEEXPw4liXcAmoOpBPF9BwLWzR/AIV6LbqV4ntcOsT42NnFQEBtYMCf/B4wbekuEAgEgiKlxJZW+fj4QKFQICbG3NYaExODgABpu/5HH32EUaNG4YUXXkCtWrXQr18/fP311/jmm2+g00lPFN577z2kpKT899+9e/msDgsKhesxaZDLGGb8IALcHXE3PrMIzkpQ1Ihr0YLoi+YijoHQTcCdw4X72o6eQI8fpTthtX0PuHeKQZlxee7IVm8wmNhUxAEAjRpYMYIBmqakR/PxOweAih0hiZ2K3WOyU81FHAOX1rAlcfcZgJMv0OxF633cg4HWb4hV5UdEXIuCAiM9Glg11lzEAYDMROazdPoC8CwDLBtiFHEMpEUBm6ezpbitY68Zz3bjN3YCzV+x3ufiaqDXT+blmAY6fQZkpXAfmYxZXNEXuE1uB/T/88HloYVMoV6LmizgwAzrx6MvAHcPAfYu7PQnEAgEgqdCiXXkKJVKNGjQAHv27EHfvn0BADqdDnv27MFLL70k+ZzMzEzI5ebalUJBy73e0MbWApVKBZVKpPI/TW7GpsPP1QFKuwfrjgFuKiRn5SIlMxfuTqKN8LOEuBZNyMkAjs6R3manYnvdCu0KvuW4KT6VgUn7gcvrgLC9dADVHswyptRIYOvbbNvr4M7si/R4QOkGaNV02WjUPE5GPFuDm7YLTrnP93h+OTBwAcsrDDkWMhlg5wT0+IGByRdW2D7Hi6uB6n0YutzqTaBab7qWspKBGn0pErk/oDRDYIW4FgUFRsp9Ou+kiL7IEsy468bxwpLb+yj6GFqSmxJxluWdAFuHt3oDGLwQuLKBZZsVOwLBTYAcNTByDcexu0c5JtQZSkfOpdXcv/dsIPk+UKEj4FMJaDgOcC9Nt85TpFCvxfP5jK2n/wHqjQJg3SFWIBAIBEVDiRVyAGD69OkYM2YMGjZsiMaNG2PWrFnIyMjAuHHjAACjR49GqVKl8M033wAAevXqhR9//BH16tX7r7Tqo48+Qq9evf4TdATFj5ux6QjycHioff3duN/dxAzUdvIoxLMSCJ4iWg2dKAbkdkDTaUC5VpwUOXgC6tTCFXJkMq6Ut3gVaDwRgIylDFmJ7PIyYQeQnQ7kZrCE6oWdgDabna4yEzhJ0uu4Eq5QMoNB6Qw4+9NJ0+dXOmWykoGu3wE5aYCDB/dRKAGVK1fxTy/Iy+PRWp9jdiqQdJehne6lAK+KQPcfGdaclcztGXEUhjJi2dnG3glw9QdcAs3LvQQCwcOTk8lrKzWCOVSugYBLgHUeVs6DmhPorZ18lmizjYKQQsX9czIouEw+RBFIkwWo09jGvN3HgEIB5GbS1SO3Z+hxvdEstdRrAcg4ltUeClTrwxIqbTbgV53lRtlp7MznFsRtKtfH/6yKK2oTgU2u4PeMwTmVkwZAllf2Ku6fBQKB4GlQooWcIUOGIC4uDh9//DGio6NRt25dbN++/b8A5PDwcDMHzocffgiZTIYPP/wQERER8PX1Ra9evfDVV189rbcgeAhuxqajTojHQ+1rEHLuJGSidvDDPUcgKHGoXOk0uXec7pZ+84CrG4ElPxv38SwHDF9h3QGmoJHJKK6kRgHb3qZ7ptPnwJoXOLky7FN3JFClG0umDA7IXr8ASbeBIz8bV86dfYDuPwCn/2KoqIs/g4w9ygIbpgGl6gOlGrArVq1BLOXSaTnJOjIbiDpnPLfy7dha+O6RvBDksXxs5SjjPv3mc+X/+ByjGOTiBwxZSifRg9qjCwQCkhHHTkbpMRRE7x0HDn7PrnAOHsDAv4AyzSn0GvAsx/FByhXt6Mn/AuvYfk33YF63G16iqLxuEpAey21yBfN1AutQdN79CQWbvnOAKxuBvV8wDBmgoNzlK+D2AZZlymRAjf5A1Z50Fa6dCEAPtPsAWD3WWA4qkwMNxwNt3gVcnm5eToFTrQ/HxmYvUYzXZPH3eGUD3//ltexA6F/jwaHRAoFAIChwZHpbNUUCSVJTU+Hu7o6UlBS4uYmQt8JGnatFtY+2Y2Kr8mhX9eEsvJMXncLEVuXxcodKhXx2gqfJc38tJocDf3YCQpoyj+bUn9b7uAYAL/xrXrZUGORmATs+4DkMXgRsekW6XKLFawwVvbmbN//1RgHb37Xez84BGLIIWDKIPyudedyVoxiAGn+D+RcnfjcKQE5ezO05+Sdw5yAFoN4/mwd1AkCbd4C7hxnUHFSfgtjuT6zPwd4JmHYU8Cz7JJ/Mc8Fzfy0KgOR7wKoxQMRp42PBDSlwrB5P95vcDph2jE4ZA+o0YO+XLHm0pPdsoM5wIC2S48vVDdb79P+dom75NsDyEXTZWNLpc+DWv0CDcXTUJN5iuLIlMhkwbDkzewwCT7OX6dA5NoeByeummrshDXT5Gmgy9am7+Ar0Wky6SyFn0yt0UQL8HTZ6Aag7HFjYh49NPgh4hDzZawkEAoHgkRG+cUGx5nZ8BvQAgjwePozUz80BdxNF4LHgGcejNDB+B2+qzy+T3ictGki4Wfjnkh7L4GW3UuxAYyvz4vTfdNEAQN0RwMnfpffTqNm1qlQD/pyTwQlFo4nMsLB3BI79ZhRxAOZkrJsMtHwNqDOM7Yy3vm197FN/Mv8CAOqNtH0OuZks9xIIBPmTkQCsmWAu4gDA/VPAoZlA06n8WacBzi01d984uAKt3wR6/wK454kBftWA4SuBar1YApVwC6jSBWj7LsVpmQwIrMvOSeHH2K487pq0iAMAJ+YzH0ujphBh65rX6+nGqdLd+NjpvzgOeZWnWCUl4gDAoR853j5LqJMZFm0QcQD+Do/P5fjsFsSxPurCUztFgUAgeJ4RnnFBseZOPOvnA90fLiMHAHxdVLgnhBxBSUarpeNEo6Y7xTVAOlTTsywzC2xNYAAg8TZXqwuT3Ey2F3fx5yquLdTJxg5RroHGAGMpku6Yd4RJusO2t3Yq4PBM6edosoHoS0Drt1gioXJhKUfdESyd0OYwvNTQMtfZx1j+pXID6g4DyrQEoKcAlhLxUG9fIHiuyYxjGZUUdw8DLUy6RcVc4nVoZxLQ6+wL1B8NVOpMoUChMi9TSgwDrm8DmkwCKncFIKOwu+czOhNrDaRLzxYp93mtx12jAJR0h7k91XqzdEomp1hx+m+Wh9UbDVTvy+fGXwOUriwBS420/RoZ8YAuJ9+PqcRxY5d1NzEDx+awpGzHB/yM0KNIT00gEAgEQsgRFHNuJ2TAWamAq8PD/6n6uqpw8k5iIZ6VQFCIpMcCZ5cAR37iaqeTF9DyDaD2EOkMBntnoPnLXDXW64Hb+4GLq4whor6VC/+clc55OTn3AZ+Ktvdz9mVIKAAk32VHmpjL0vv6VmVHrP9+rsIJQ5Vu+QtA0RcZQhp9idlB909ywpceS7Gm8USWV1TsyCDWgNqcQNbsz2yPq5vZCcu7ItD5SwY4a7K5Ei+TsYxNtCsv/uh0/H3qcikMuPo/7TN6dlGn5b/dVAwIaWIu4phiq5V32RZAUhiwegLzdpx9gSaT6fSJv8kcnVL1pctLAcC7Aq91v2p87YA6bGt+bRuwfioFc48yQKvpQKmGbHt+azfH06D6QLl2QLv3KAQZkMmAcm0pIildKADJFRx3lc75fx4lhfjrtrcl3QHKtgYG/JEnckXxWrN3Bpy9i+wUBQKB4HlGlFYJijV34jMQ4O4A2SME6fm5qhCTqkauVleIZyYQFALZ6cCBGcCeT43lSZmJwM4PgCO/WHd40enohom/yRyKtRO5/5DFLAXwKs+b7MLGxZ/5EOmxgJ0jLfdSNJnM0goAOLOQz5FC5UaxJfoif3byYq7G6b/YZti3qu1z8avKVfqAmlzF3/mBMfw0OxU4+AOw9ys6AHxrMEcn/jowrxWwZCCdUEMWsdvWor6cvK2fBsyqCfxSn+VayeGP8ykJior0OODkfGB+a2BWLeCvrgy3zbRR8id4MqRafxuQyYzCjb0jUHPAox1bnQocmsXxT53CxzLigFN/URTKyQDmteY52OrS1/RF4PoOirOabAo2h2ayHNTQ1jz5LrDpVTqLMhOM5V+RZ4D0KGDlaIbMu/ixPKvffCCoLrDrI2YDnfqT4vG55SwFexYIaWx7m281Zp1tfIXCd/x1/h6WDGS52wO7kQkEAoHgSRFCjqBYczs+479OVA+Lr6sKOj0QmZxVSGclEBQSGbG2V5WPz+EE1ZTku8AfHShY6HUsS7i6EVg7Cej1MzByLdvtFjZ2KjpdGk5gcGnv2SxhMGDvCLR+G/CpAtw7xscSw9iKvNsMY6kTwMnWgD8otgBcRR+xhuHDwY0YZtxWIiAZ4HECagNR54Fag4Hj86T3u7yO52RnDyzqx89Mp+V/oVsoinX9liGn+74G3PKcApps4OxC4O8expIsQfFCnQLs/RrY9o5RwEsMY1D2lfWAVpPv0wWPgbMvULmb9LbK3YA7h1kGOmaTMQfnYcmIAy5IZIB1/x+wYhRwYTlLtfZ8DvSfby7yKl2MXe06fkoRd+0kBhnfPyn9egd/YHc7A37V6D5JjaRTp+9vQNdvWKJ5aCaFcwCIvcoAZTt7YPP0/EtMSwplW7FjlxRNp1IIy81kBtG2t/m5RJ6hcBp5rkhPVSAQCJ5HRGmVoFhzOz4DrSs/WktPP1cKP/eTslDG+xmxOAueDzLijS2wLdHmAlmJAMryZ00OcPIPY/imQsm8B9/KQFYyEBcKlGlRuOebncbz0OUC0LM0ovNXPNfeeS3FtRo6auT23G/yQSAnk8KTgxudOb1/4cq9QsksC5U7J17aXHaYyYij26jDx8y8kdsBPWdyYmVYpfeuyPbB/36Rd3J66ewgpQtQox8nn/E3eB6WZCWxy0359lyhbzoNiL3CNuYAHTl3DgN1hlg/V/B0yYgDzvwlvW33p0ClTmxZLSg4HD2Anj8C21XsLKXXM3emej+g/fscI5q9aLt0Kj9S7lu3Jg+sA8RdZ7mUgbhrwObXea0G1KLwq3TmmKJ04lhStQev6fBjtl8vNcLcYeRXHbh3gv9ODAOOzmGHqox4jh0395iPIfu+Adq+z/LWlq9LZ5uVFJLD2RVs5wccK30qc+wMaUxxK+m2cd+4UI71bqX4GW57Gxi9nuOsQCAQCAoFIeQIii2ZORrEp+fA3/XRHDk+LkrIABF4LCh5PCh7xc7kWlAnAzd28N+lm9Lxcmk1cHk9J0whjelIcHuMydODyEzijbwmi6VS90+yvKrRBGbeKJQMQD2/nCvWHT+hm+X4XIok5dtTBLm8gUHE6hQgOyXveclczb62hblA906yRKz3bE7MdnzIiULdEcCE3cZsiqhzeWVPeSvhCnvr8645gF2zzi0F1k0BvCoAfefQhWMo+TJw5xBzdFLC+V5LNweav8Lyi/QY4Mo6TmrslA/+vLS5FJ8eoURU8Jgk3rae+BtQJ/PvTwg5BY9bEK/RDh/zM9ZpgDsH6U5pOlU63+thcJBooR1Yl9enJSn3gR3vc/xr/xEQfhxoOoVdleTyvFKqXGPXOinkdoCjl/Hn7FS6iOR2wOCFdPNseoUidrlWwIgpwPZ3jWHLqZE851ML6FB0slHuVRKQ23G8a/YSUKYZS11P/QmEbgYqtGdnsS1vGEtNw4+wpDU1gqHW2elCyBEIBIJCRAg5gmLLvUSWRj1qaZWdQg5PZ6UorRKUPJx9KVokhllv86tGt4oBhT1t7+7BFBiWDzfmPcSFsnV267cZhCw1GXpcstOBi6sBj2Bg9ThObAyveXs/X8+3KrDhRT7e5h3gwkq29TUQexU4txgYsxlYMcI6vLhSZ6Bca2DZMAotLoHArT0mbhswm+Lfz4Ghy/gZ7PrYPFQ1+hIQ3JAtkAHmaZRvCywfZpzox4VSMOo+gxMw03bjDu7Meag3is6n6AssD+n9C7BsCODkk/9qu17PCc71bTyuVwXm8rgHPzthqMURlUv+2xU2gnYFT46DK5AWyXJF0zbdt/fzOur0Od15j4KLv9HlYSAn3XbJDwA4eADeldgZ78hswLMMnTIGmk6maJ4rcY9QrRdLtcq2ohAVtp+ZWUH1gPMr6DgyEHuFzpsBfwLLR9ABKJPRjeToIS0mlyRcAvhZy2TsTnV7v3FbzGW+9z5zgKWDKdw5eLBFO0BRXi6mGAKBQFCYiIwcQbElPM9R4+/26DfePi5K3BdCjqCk4RoADF1qLtgAnMwMXsigTQOOnhRwGk4A9n9nFHFMOTjDmBNSUKRFAdnJwP7/SU+EjvzC85TJOVkKqGUu4hhQp1CYqdzVetuNnVzhV7kBOz8E7B2YVWOJNjdvNfw6MHiReTec43OZc2PI5Gg8Edj9ibRbY8/nQINx5o/VGsTAY0cvijgABac7B9mtpuH4/IWcuFCGf257h91xjs4G5jThe8uV+F0JCgb3ENuht0H12XVMUDioU4GdH5mLOAbOLmJXu0fFLYjOD9Pf6c3ddMPZos4wXqtH5wA1+jBA3pQjv+SNFxaLRH7VgbrD2cmq0QQ+ps1hRyavCuYijoHMROD0P+xeBQDl27F0q9lLDxYVizt2KoY6q1zNRRwDadHMnaqa13q8fBsg/Cj/XXuo9feYQCAQCAoUIZcLii3hiZlQ2cnh7vjoq1reLipEJAkhR1AC8asGTNrPFc+4UE4u/KpJl4OUbsaV6T2fSR9Lrwfun8i/JfijEnmOocWRX9reJ/oinUXOPsYbe0tkMopMzV8Bzi9jqUJgHa7kxl4Bbu0FSjdhmVXsVevsIIUSCKwNeJSlkyl0C0utwo9xFdmzDCBTAJ2/4Eq5s68xmNSS7DTzsqfaQxmanB7LcOOQxhSw3IKB2FC6jvLrBpaRAGyYxkygkCb8f9R5vod1k4EXT/L8BAWPayAwbDk7jpkKjS5+QP95ojVyYaJOBm7utL09dBuvq0fFvwaztaIvsqyxdFPmaLV8Azj0g/m+FTqw1by9A6BOZBmoziLgOvwYIFcCg/6i4JMex9fQaYHj8+kacvQE2n8MBNVh5piUiGMg/hrQ9Tsg+jIFnBu7OJaVdBJvATFXjEK2FHePcIyt2JljsE5DR2bbdx5cKiwQCASCJ0IIOYJiy73ETPi5qR6p9bgBXxcVzoSLVrOCEop7MP+r3CX//Vz8jG3KbVEQ9nathqUN6bEURewfUO4okwPQG0NPAa7uGkqfag1k/s39U0DYXqDfPL7f0K2cDDadxtVyJx9m7Zi6jWQyoMVrLL26d5IiiU4L1BiQ1/JWBlTpRuFErqBjR6cBes+xPg9TlC5AkylcXc5KZtlAuVZ55RIKoEI7Ph5/DfAonf8kJSsRqNYHaFGeXbrsHIA27wLXtzM7Iy5UCDmFhVwBlGoITDtOF0HsVYppwQ1FNk6RIANgI6PocYN/ZTI643yrANps4PxSwKsi3TNVewCX1/CaLtsKSLnHznY9ZzJHy1aGVU4qsGIkyyU9ylIw1+UC3hWAsi3ognQPAWKuAt7lOQaYYqeiKNXqDSDhJv/WWr8JeJRhV738WrKXFPQ6OpJkJuZ9mYzB9fZOQJcvWap4/xTLsOqNAurllY8WRbdEgUAgeM4RQo6g2BKekAFfl8fLM/B2USI6RQ2tTg+FXASMCp5hnH1YMhJ5xviYezDQcjpLteydGcTp7Pvok4u0GLZET49jK+6cTDpt7p0AghvT7WOJTMYwY9+qdNXUGsROU9lpXOXOTKQgtHSweZlTcCO6c1aNBo7N4Qpv58+ALV8D7T+kIKXTMMQ06Q5zOAwcmMHXqdYLKNuMEysHN04yqvej48fJCxixGshM4HnEXGb74OxUupr0egZ03toL9PiRr7XUpCvVgf9x0lipMzCnKTDoH8CzNODgSQeAwmTCKJOzBGv3J+afTYvX2L1HqpuWoOBQ2FEo8xz9tM/k+cLRkyJq6Bbp7VVstCh/EOo0dkha3J/h5gZUbiw5rdKd5aUXVgLupTheLOhKZ96w5cyq0eaaH1Oj5piQEQf0+IFdlhJuGbfv/RoYsgQIacicnRp92WGv5WsUcHIyKOjeP8mW5QZB3T2YbdafBSHHuxJFe6+yHOtbvs7vm5xMju839wB7vzC6JeUKlqwF1Hqqpy0QCATPCyIjR1BsCU/M+q+V+KPi46KCRqdHbNoTZlFoc4H4myLTQlB8cfJmq29DgK5HGaDXT8xkWT4cWNQHmN0Q2PQaMw0elvgbwN/dgbktgcX9gCUDgQvL+XqBtSmoKCUyIJq9zM4mK0fRMXTnEEOL104Clg3NCyb+yDqr5v5JulfKt+fPjm7A0d9o6z+/DGj7HlfJXQM5obIkLZqi1T+9uNK+bBgwrxXLpAYvBLa/w/ewbjJFpOvbgIELOBns9Dlw6Eeea0YsTQXH51q/RugWijDeFYDVYxmQPKcpH8/JMO535yBzPCw5PIvlcP6PUV4iEBR3VK5Ax8+lM4oaTwJcgx7vuBkxwPop5iIOQBF2/VS6ZdQp/LnFaxzrctK5z7klQLsPrI958k+g189AneH8t6mIA9CJsmo0ReFKndhufORaCr1LBzPo/Y8O7Mw34E/je065D2x4+cFOyZKATMauVFoNO5Id/JGhzmsnAvNbc2zu+5vRsaPTAitHswxVIBAIBIWOEHIExRK9Xo+I5Cz4PKYjx/C8yOQnEGBir3ICPLsBMKsmb+AEguKIbzVgyiGg+atAl6+Bja9Yd766sg44OFM6FNmS1EiufifcNH/82lbgxHx2kdn9CQWSRi/QEVS5K4URhT1w5h+KoP9+QaHEkCcT0oTlRbY4txSo2Z//rtIDuLiS/768jtkYQ5awY5YlMhnQ4lVgxXDzHBxtDnD3MDvWRJw2f869E8Dhn4AJu5ilkpsF1B7MydrJ3/M5x2VAjf58fwk3KVatHmf8vNNjKaLZInSreWi1QPAs4V2BGV9t3wdK1QcqdgJGbWD3usdtxZ2ZRAedFGlRFHBq9OfYY+9oLvhc3cROSkMW07EXVJ/d41q/xXGu0QTbY1JuFp+76xO6bw7+wPHElKhzHOdavWF87O4h5mSVdLJTKejbO3CMS75r3KbXA5fXApFnzZ1WOg3bkAsEAoGg0BGlVYJiSVJmLrJytfB1fczSKmeWOUQmZ6FBmce4ecxOA5YMolW4wyfA1c1chRu3HQhu8FjnJBA8ETmZdItkJhjzY1z9uU2dzFiKeiO4ctzje64wH/nZvGvVmb+BZtMenM+SdJets6U4twRoMpmlXEsGcgW8xavMutn6JoUUt1IMBPYoTZFl2DKev4M7cPRX26+rTmZmxdCl3HfgAjptjvzC9uP1RjFMtP5oTi5u7mbmTKmGDN3UabnyX7EDXXR2Kjp4Ds6Qfr07BzlZKdOCYpizN0NQM/NZTc9KNLqfMpP4b72e76vHTJ5DfqvxmfG2czsEgpKOTMbxpeV05mCp864FjZrODoUdr4+MeLpoHNwfXPb5IPE5O43lPB5lzJ1xABBYFwhpRNdIg3HsgnVpDb/PSzWguKzXmT/H3gloOpUh5/bOwMA/eS/QcBzHWJ2GY4tOy+OeWwL4VDI/xrNQPpmr5u/G2Y9lqXKF0f2k01DEObWApWym5XSpUQyFN5S0abN5HNdA81B5gUAgEDwRQsgRFEvuJ/Em6HGFHCelAo72CkSlPGbnqsM/0wHQZw5zRgLrAjveYznFtOOA0unxjisQPA4Z8XTCHJpJlwnAle/Bi1nOEHmGnVLOLzXmFQTUAvrOBTa9Qrs/wAmR5iGuiZR7trdpcxj661GaYk9qBMN7r23ldq/yQPcZwI73gbhrfExhT4GlfDsgqC5w2saxe/8KhO2jAGUIJPYqT2HKoyywbiIQkZcFJJPTvdNvHl0uyeH897WtLKsyTM6CG7L0Yd0kfo6WZCaw5MtA0xeZgyPVbhdg3k/0Rf67VH1mRrgFUUhLCeekp1wb6ZbrAFCtt3mbdIHgWSMnkyLp+ilGh5yjJ79PA2oCm183Lz2s1BnoOYv5NlI4etFpkysxdsnkvP4izlBg9q1KsUCvByp2pJi0432jS0flBrT/gEHIMZeZveUebBwj7Z3Yzer4fDpwDFToQHEqJ53HM92/xauAg4dRuHD05HEzEkp2lzQHN+YH3T0CeIQAtw+wvPa/75ja3G6ZP+RdAfizE383HT4yhtoP+JPimEII2QKBQFAQiNIqQbHkfl7r8McNO5bJZPBxUT5eaVVOBnD8N5Z2uAbwMYU90OJ1ugNMb+4EgsJGrwdCNzPM0yDiAHTc/N2df5P3jgNnF5m36I6+CGx+zTwfQunCFeYH4VXe9jZ7JwB6oNWb/Dk5nJMnA+0/Ym6FQcQBeKN/9Ffm4HiEWK9eA0BQPa7yHphh3lUqMQxYPYFhpwYRB6BQc3E1cPsgz7daL04WLq0xX2G/fwrY+QG7Rlkik5l3ZAGAY7/yXKTKn+wdgTrDgCvruZLv4s8yspWjga3TWWqQlcQyEimxxi0IqNDW+nGB4Fki6Q7FUdMyx6wkCs6bXrPOj7qxk2OVlJMtPQawc6SIIkXd4RRO9n4FbH2L5Zh1hlNIaTKFY5FpqVV2KrDtHYo8Tafyu920LKrZNOD4PDoATbm1h9lZq8cZRRyAzpt93/A83UP4WKs3eT6JYXQhlVQcvSnKRZ3jf+eWWHzHXKAo5+JvfKxUAwrm2al0WG55A6jWkyLcon5Akg2np0AgEAgeGSHkCIol95My4WivgLPqMduVgp2rIpMfw5FzaQ2Qnc6bD1PcSwHV+wJHfwFSIh77vASCRyItCtj3rfS2rCSKEaf/kd6eHM4JjYMHf272olGczA9nX7b6laL+KJZ1Obgy6DIzgS4VeyceOzvNvJzLlJN/cP8u33Cl3CB2BNVn8OjB76Wfp05m228pAejCcqBsS8CvGrvWSBF7ledmuRJcuSsDji3Z+CIweiNQtZexZXK51swEOvAjW+x2n8Fg5dir3J4Rz9/Tns/YDeeF3UCZltwmtwNqDgLGbjVO9gSCZxGNmqKtZbkSQHecpUBi4MZOa8dcZiKw/QMgLQKoNZAh7m55rh1nX4rUzV7i9WUoZQrdCjScwHDe0C0Uh6U48w+gyWGGTm4Wuy35VGL3vFv/Wu9fqgFderZKpvb/j8LRgD+5GBS6mUJHho2xsCSQm8EFhJoDpAPmAebmpNzj+NpkCkUxy259ZxYBNQdyIeL8MkAn8bchEAgEgkdGlFYJiiURSVnwcVFC9gT11F7OqscTci6sAgLrmK8yGag5kMGIB78Hes587HMTCP5Dk80VY72OjhknL/Pt2hzbXUCULnyeoUOLFMl36QSpPZgTHIW9+fasZK6eyuTsSGXvyH93+YbumKhzvJGv0BFw8eVzLq3lCnlGDK3yMgWFj8OzgESL7i+mZCZQTFk+HKjeh6VfcgXLppTOtnN5AHaPcwtm+KYpmmx+ZpmJ5o4lq9dO5Mp9egzfX/XevJ7XTLDeN/E2kJUCtH0XaDCan7HcntlEXb5gO+SlQ6UndVfWA23fYdnB0MWAOu+zdfQCVA/hhhIISjI5mUDMJeltD+r+qE4x/zkzkQ4+R09g16d02ozewOtRJgOiLlBM7TmLYccxF+mG0WZTlDm7yPZrxV8HshKAKxuAPr9yXGz7fp7jUAL3EI5Btki4QbFH6QL4VQfsVXQmmTpYShp6LZ2QcoV19pAp6THA8FUUfv7uYf6e7Z34eVTtwe8RlQvLcR09+W+BQCAQPDZCyBEUSyKSs+D9mGVVBrydlTgX/ogtQNPj2HGi6TTp7Uon5nKcWQi0fJ05IQLB45ISwTyYMwspCpRqCHT7FvCvSUEFoPDhGmDdOtyrPND9e95gK51t32gH1gEC6tB5YlpGpM1hIOWO97jNTgXUHga0fpOvufMDoO5IZiAcnsV8Gp2G+S/tPgCizgA+VQCnEJ5b5Bk61iyFIlMcPfk+tTnAxVX8T65g7pReZ55VYYl3BeuOMYbPR5vD4xoyKqTwCAF6/0LHgELFcrCFvc3LuAy4BvLzOLOQn1/yHa64AxTD6o0C0iJNjl2agpY2lxk/90/lhYT6SrdiFgieVZR5E/eoc9bbDGOaLRzcjf/W5NBxeP8kyxkbjKGj78YOlptW6szcLSdv4Po2lnTGXaUjLu4aHX++VaXHDADwrgSUb0/X3KEfgSbTgIhTHN+kSI0AytvYBgBeFZgL5OTLnC6/6kCDsfmPh8UdmYIB0nodBRlbbiQHD2Dv10CtwUYRJ6QxUKoRUHsQcPgXYEEXCkMV2vP+6tY+hkd7V3jweRhCsgUCgUBghiitEhRLIpKz/us89bh4uyiRmJkLde4jrIjd2MmbxJCmtvep0oM3NYd/eqLzEzznpEWxc8rxucYb5IhTwJ+dzVvtugYCrd+2fn637xjge+pPCgtSuAdzVXvpIODeUfOb4fibwB8djKVFmmx2tfqnF6DLBZpMBbzKsnvbhZUUKfR6ChULe/Ocr26i+LnhJWDb28DaF3gsZx/p82k0wbz8qXJXYOQ64Oxi5su0ekv6eSo3oHxbrqJbUnswILMDoi/TYSOFT2Wuji8dzDybDVM5Qew1y3w/e0eWb3T6nOJVZjw/s3JtgBp5x67Z39h1SuXG8rKW0znxtFPRHeBTCdjzVf7dqwSCZxE7B6D5S9bZUwDF3nJtpZ9XoYP5uJEYBvzTgwsreh2dc9e2UijQ6+iMXTMB6PI1EFCXLa+XDzdmc11ey7bYhtJIS5q/DCzsRXet0gXYOI1CROQ5aTHn/km2UrclRrV+Cwjbz/Gi5gB27trz+YM7bhVn7ByZ93V5HUUpKdxDAK9yHPu8y7Oj4IhV/AyT7wDnVwB1h7GToF4P3NwDrBwDVGwP/NWNHRKlyMngQsPOj4BVoxk+nXSXxxAIBAIBACHkCIopkclq+DxmxyoD3s58fkzqI9xI3drDSVh+rVDtHdl55uwi21kgAsGDiA2VLkHQ69gVxRAUKpOxDKjF68bV3bItgfCjzJQI3UzXSO0h5pMn36rMndn7FX9u9rLRHZKdBvz7pXQpUtJtTlqq9wWS73El2pLcLIovMZeZj3Bzl3Hbv5+zZMo0MFmuABpPZIiwIX+iTHNOeBb3o2ii11EgaTrNfBXbPQToP5+5VaYTLJmM5VkVO7I0zcUHqNyF16ZpSWZgHaDrtwwkNVBnGHBhGcW0OsONj/f6GTi/HFg7kSVSl9YAaycBR2cD7d4FOn/JSV9OBlC2NdB3DnDsNwa1Xt1IkWr1OODsEsCjlBgfBM8nnuWAIYuN2VwARU//mkCfX6yFkvLt6Jb7b3xKp8NDk03XzLkl0qJoVhIXXyq0Bf79wnybNpcOnj6/mperKl0ott45xFLPO4cohMffYKZNUD12oSrX2vx41Xrx/Qz4wzxnzM6BbiCZnOOPygVYM56iePU+QFoMu1eVRHQ5QNlWQKUuQK1B/M/0O8avGjBsGctH649i7lDdkcDq8SzLDd3CxhHLhnBcLt2Mz8tJZ1B9mRbAhRWA1mKxLVfN3+tvTelYDd0CbHsLmN+GHRIFAoFAAECUVgmKIRnZGqRk5cLnCUurvPIcPVEpapTxfohsCp2Ok8yKnR68b9WeXPE79hvQ8ZMH7y8QWHJjp+1t945TLDBMQJx9gTZvc1U0M44Tig155X96PbDhRaDheN5U56pZamWnogiRFgXUHwuUbsrcF5ULJ0C22msDwOX1vHk3FWgsuXOIk6TwY+aPJ9wCtkwHmr/CgHClC907dw7x3CYf4ETN2Qf4vb3Riu9XnW6Z2oMZKqzJoaCTEccyrwod8kqjcrjKrXSmG+nwT9zm4A7EXmGZ07AVnHCoXPlZrp3ISRvAluFlmgPH5rAsq/3HQI1+LNGIOkuBzJJr24C6o1hadmMH4BJAcejSKk7+LDm7CBi4gK4nW6HRAkFJQKMGcrNZMvWwZUJKJ44fUw8B6fEA9BzDXAN4jEF/U4RWp/C6dfbhtahOZcvr7FTm3PSbS1Fom4Qj0cCdQxRmDaWncgXLrrwqMGj4zCJgxBqK1jkZfC1NNkX0plPp8lC5Aa3f4bVfuQfgX42OR20ukJMF2DtwAWfTqzzn9h9xsUer4fu5uIrCj18N4N/PeB73TwEJNwE9OAaWxDbk2lz+F9wAmN8OqDfC+B1jp6TLcc/neYsMdgyB3/42FwpM0WmB7e8ygNowvt49DDQYx4yiRi+Yi23pMfzusnTfZCUBG18Ghq+0zpITCASC5xAh5AiKHVEpDCj2KYDSKtPjPZC4q7xRCKzz4H1VLiwLOfkHs3Ic3J7gTAXPJVLtrQ2oXAG5hWFS6cRSJ6+ynJA4mtzI6nX8Wzz5B38ObsS/y9pDOMnJiAd2fsy8gvgbedktHrZDkl38ObFyslEiBXD13N6JK9KWJIcDW9+kmDJ4ITs7BdTihCcrCajUFShVz/z1Ndmc8FxYKd19qvYwCkAHvwdS7wPBTYAK7Zip4+gB7PqI5Q0Btems0+uABuPpAqg7gq9VuinDWA2TBEdv5kAoXdn55vTftt/v6b9ZjnFzJ1eWtTlcVbbFtW1A81dtbxcIijPZaSxvOjoHSAoDghtz4u1RBrB7CEFHYUc3nVSXNicv40Q8LYaT+pN/UDRoOAEIrg80ngyc/otjmamz57/j5+VhOXry2pXJWBLd+i06OO6fYDh6s2nserd2IkPKJ+wCUiP53nKz+D1upwTKtgBOLQCO/kSnSPW+wN5vgKZTgL+6sFzIIEJseNH6fMq3BcJPAFW6M+tsz6fApdUUL27t4bitLGFh5woVx9yk2xTjT/zO/yxp9QbFdic/oMlkOqz2f2e+T3YaS3btHfm5O3gwHNnRg69jSuwV28H190/SrSqEHIFAIBBCjqD4EZHMUiiDEPO4qOwUcHWwQ2TyQ5ZW3T3C1byHXUGv1pvlFGf+4QRPIHgUKncDdn8qva3BWMDZQuhJj6M7JSuJQkz7j4DbB6QDKFu8yi4hwY3YtSo9Cmj5CrBiJB0zZZoDdYYCB2y0+64/iiVT1XrR+i5F44m8CfcsQ/ePVGhw5S68IU+5ZyzxAijsdLVoqX5lA4WnU39Kv17FDsC8lsa2xvdOACd/Bwb+CajcKcjs/x8nCqUacmK39W2gcmdO6uwceb2atjiuP4oTrtpDAJcanGBY4luVn6ezLwA9yzD0ekCrBjT5iMS5mdKd76TQ5tJRkBbN9+cWyOfamUxwdFq6q9Jj6EpyC+I5KW102REIHpdcNa+V9Sah/4brbcxmBtkWBGnRwLopQNhe/iyTAx0/Y9mmYWyKOA20e48OQoWS7cbLNKMzRunCDDGVO0skq3RjTs5/uTQn6Jzt+RMwaDHg4g3s+pgiq4Hwo/wO7/otcHUDr+17JyhYDPqH7iBAemwAWELa4jXAuyKdPSpXChat32arbQcPCjkNxhTMZ1aUyGQUvh+U9ZWTye8sB3f+7VTowM9l3WTz/TRqdv9DFr9/Ti2g88mym5+tUGUDtlrKCwQCwXOGEHIExY6o5CzIAHg+oSMHYHlVdMpDCjnhRwHvytIOAymcfVjrf3QO0GRKye5OIXg6dPqCThJTAusyUFuba/ybSroDrBhlXsZToQNb8f7TyzxQs9ZgIKQJ81n2fA6cW8wVUr/qFHEAipaNJnLl2bKrS8dPWQKx5S2GWDaZwkBmUyp2oBtn71c8Rs9ZtLyb3mB7lgMaTaKQc2ah+fOzU3n9yBXG0qrr24AhSxj4HHXeuK9MBvSYyXMwiDgGNGpm/bR6E+g5k1k1uVnsHgNwclWpEyeKhmweAy1eo1BSrSdw+yDgVoolk4dnGfcJqg+0fgPY8oZ56UbjyVyxr9yVk04pKnTIvx26gZx0BoBueNFYkmDvxG5hVXvS7afJAe4dA1aNMWYnKZTsHlZ/tFidFhQs6dHA5tetH9dkA+unAmO3Aq4PKVLmR+Q5o4gD5JU164CDPxgfS40AspLpLKzSnePZoR+N232rcvxp9QbDzKXChbe/DUzcy1JHUxHHQGIYz6NCB+Dmbj6Wkw7s+xro9j86fjLiOKYl3TZ57SrMzdryBl2IAMWo2oMpOGlyAOi56GPakaukkJPOFu1SrioDlbvR0bTnc+sxstv/jGVxcgXgW41jf5XuFNxDGgOe5a2PGVDb9ut5lMk/w1AgEAieI4SQIyh2RKao4emkhJ1laclj4OWsROTDllaFH6N9/FGo0Y+rbVc2ALVsdMwRCKS4sBJICc+z7B/nKm6ZFpy0LB8GTDlEt0V6LLB8hHUw8q09tOpPOUwLv1ZDUcI9hJP/y+u5IqzXMxD42G/mz18/Fej0GfMJ7p1guWBwI5Yj5KqB0I0UWZpOBYav4PWhyQVKN+akZdMrFFYaTqBQM2wZEHmW51uuDW/cT8xjeLgUVzeyfbDhvHRalj90/opC0s3dLB+r1hNIjTJ3BqlcmYWjTqG7x6scEHEemHaCYZj3z7A8Q6cBTsznpKL+aIpW9k78nG/s5Op9h0+Aiu04UavUicGqGXF8nTZvszOOaWt3nZb5Ol7lmQ1xdRPPwxS/6vw8Yy4CHsH5/x0k3qZAY5oHkZvJ388Llfg7SbkHLO5v3lpdmwPs/oSTySrd8n8NgeBRSLgl7bADmPuSlfjkQk5uJh0+ptTsz7HQMhtl10fAwL/Yqer6DvNtcaHMCxuymOcmhSab1+iF5bbP58oGOu8MQg5ANxDAx0/+AbR7n50CDefX7gO6TgziKsAx8fxyjsONJzKDp3IX269bnIk4w3G63kjm35y1EORlMrql/upuXiZrGCM7fQ4EN2ReUONJ/F4Zv4O/D4WS4/vNnVy4cC9lfL6LH/c/Md/69Xr8aB42LRAIBM8xQsgRFDuikrPg5Vww7hYvJyWiHqa0Ki2GK3+PGkzqWZYOiqO/CiFH8GjotcyMCT8OdPuWN7kRpyhQ9PiRq8AAhRGp7lYAELqJDpq27/LntBiupl9cCUAPTPwXiLnCjAgpN8u2d2j9bzSBN9bLhwNjt1AIMux/7DdOYkasBo78Apz9xzzMcttbQPcfeMMff4OhwSn3KC5U7Eg3kRRnF1NEGfAHA4uTwwH/GlxtVacwvDnhFnD7MODiy+eoXPl+nbyZo+DsSyeN0gUo3YjijEIJ1OzL46lT6ChaP5WTvIBaFECO/mp0D8lkwJWNXNEHgJFrmIcTe5Xvx1TEMeXQj+weNuhvCmY3dnGVueYAlq6tm8ow6PzQqOnos9VS9+APwIC/mMVjKuKYsvdrij22Wr4LShaZCcxwCT9GMTCkCcvsijJfxXKssNr+GC2g06LpfIk6z+9Nv2oc60yRK6RfW6/neHhRIjsL4HEzE+mmtdXuW26X//vSaVmeKUW51nSJ3DsJDF0GHJ8HpEVyXDUVcUw58Ttzcxw88s9DK87otWz57lOFpWGBtVn6mhrF+56WrwF3j9rOWjs+F+jyDYW/rGRg5/vMDLp9gGN3cCOGWquT+TcfcZplpYF12fY8pDHHQMPrdfiIfzcCgUAgACCEHEExJCpFDS/nJ+tYZcDbRYWz4Q+o7waMK28+lR/9Rar2AvZ+wdWrUvUf/fmC55PqvYHzS4Fes+huSblv3ObiB4zeyJXHjHxaWOv1xpvolEjg2GyKFKY0f4V5OVW6GcOQTVEnc0J18AeKIG5BLAPo8CngHswJ0P0TDL20d6TIpFCyzOvMQoot66cB/X8HslM46dTlUlS69S/Q7EW275bi7iEAecGiLv4MSHXypXD1W3PjxGvEKmbcDPgD2PctnT8G7B2BYctZMnFinnGS2fxlnueuj4F+v9H6f+eQ9Tmk3Adq9Adu7eV5NhgDVO/H0itb5w1w4qHXA9vfA4LqUmDS5gChW/k8mRzwr277+QDFu/hrtrcn3OTEVKozloGk25wMCUo+6THAtneZ62JAruDkt1pvCjtFgXclXvdSWSQepQEnz0c7XnI4sHgAEH/d+JiDO118944DMZf5WOhmoPEU6WPodbbFTICfnX8N43e5KXI7uj8qdaaDToqqPYCwfeaPBdSii2/Xx3TgtXmHmTyNJtIFaOtYAMfVnAxmneXmOVBKGkH1KHQfnglcXsPvkj6/8r3YOQCX1hjLqaRIjWTp29oJQPQlZghFnTfmpTn5AmM3A+unWI/pI1YD1foC5dpyXFU6i5IqgUAgsODJa1cEggImMjkLXk8YdGzAy1mJxMxcqHO1+e8YcZrdL5x9H/1Fghty4n1qweOdpOD5xKMM0PV/DDw2FXGAvHKq4bxJdg20fQy5gjfHAMUPSxEHAI78TAdNjX4UaSwJbsTjJIcz08DFn5OQxDDmtqweCyTf40qosy9zaFaNoWOg9y9AhfYM/nUNoHCzsA+woCsFiBs7mcdTrbf166pcgabTgMM/A/9+AWx8ibkcMtDlY7p6fmktX+vSGvMbfoCZOMuG0gkz4E/jhOnIL5zYyWTM72kpkfkBsE2xiy8nlF2+4gr0wl7AqrEskbKFZzmGD7d5h46ZTa8CW99i1oZeD7T7yDqw2hJ7JyAgny55fjU5qcmv5NO3CkUuQclGr6cwYCriAHSKrJ9qPUYUJi6+LIuxRK7gdZjfmGSJOpUZMqYiDkC33IqRDG03cHUzBYJGE6yPI5Pnn1/nVoqisVzCVdP6TSDxFp8vFdTsGsA8qusm+TkKJa/tk3nh64lhLKO69S+gdOT4F5hPlouzL+ASANzeZwxMLmnYuxg77yWH01lz/xRdmsl32TUwPxezZ1n+3kOa8mffynyegUod2YVQakxfMpCuJxdfll0JEUcgEAiseGpCzunTp7F48WIsXrwYZ86ceVqnIShm6PV6RKWo4V0AQccA/jtOTOoDyquizjEYVSZ79BeRKxjSeGmNecmJQJAfzj7Mj7HltkgMY1aLs6/xRtiS2sMoImYkAMfyKeM5PpfCypAldKr4VgEC69D23vI14Np2YPAi3phnxAF/92QnF42aE8wqXTnpOrXAWGp09zAfazKZYaNrxtPxY3AIXVnPzIt937A0occPFI28K7It+LjtFCDKNAe8K3C1fNRG4Nwy6/M/vwzwCGHujxS5WbyGr+0A2r5nfPziKnbeyojPazNuUZ7iVZ5ZFte2c0V+1Rg66wAg8gwnEY423Adt32XQ8qU1LGGrOZDvo3xbYMwmoOHYBzso7JRsb2woozNFJgNaTQfsHejesrfRnarDJyLs+FkgPZbioy3OLy26c1E6A3VHsMyyQnv+XdcaDEw+aHssskVGPHBzl/S2zET+Xff+le4X74pcVGn2MjtGlWnBx+qNpCjbUELgAQD/mpz0x19nBkv1vsZrccAfdL7tyyvxaTSRIlVgHTpwm70EDFsJyOyAkGZ8vTrD6fI7Ps9ceAA43h74nu3EPcvaFrUaT+I1fG0H3UIlkcx4oHQzlpO1epOi4sHvOVYuHUz3WPk2tsfIptOAAzP4vQIwq+ycyd9xlW7MJpIiN8s89F4gEAgEVhR5aVVsbCyGDh2Kffv2wcPDAwCQnJyMdu3aYfny5fD1fQxHhOCZIVWtQVauFl4FJOQYjhOVokYZbxsZA3o9u2dU7Pj4L1SxI29QrmwE6o14/OMIni8e1NUoJ4OCz8A/WcKUlcROVtd3sOyhw0ecdGUmGQN6pciIB+RK/o1mJrDVroMHW8Fq1BRaNr0CvHiCYZ+mkxfXQLpjYq9YH1enYcZLpy+tb7rPLgb6z+f1te1tOl/avkvxSq5k0HP4UbbzlttxpfvCCrZKlyI9Nv/PKycDCGlEh5xXBU6iIk4bnT0yOdDxc8AtgMHQSida+5XOdBvt/846+2PP53wPOz8E4vJKoJTOQOu3OPFs9QZXi+1dgICazPUJqMWMoYfFowwwYg2wfrKxTMHZh+U03hX5s3sISxBWjzdmDjm4U4jLz9EjKDnoNPwbt0VyOP8+H2ex4XFw9GAGVEBtjhFKl8drdW8Qg22RlQjUH8lJvV5HUVKuANxL0z2j01LMtndg6LBGTZHZUPZVpjnLPTe+TJF6zQs87x4zWRK6433j51q1Jx2MgXWA0s05JqRFAdnJgIMr0P5DuhavbWPgvCHwuUIHoP4oisFuQRRpbv4LlG/NUqPdnxoFeYOjSOnM3D1t9oPbdxdn1k1iRk7fOSz1LNOcv5Nqfege2/4hx68NU41jpL0T3VFZSfxclC5Ar585RqdGmB8/vzHdcl+BQCAQmFHkQs7LL7+MtLQ0XL58GdWqMbTsypUrGDNmDF555RUsWyaxGit4bjA4Z7wLKCPHIOTk24I8LYorT94VHv+FnH15w3t+uRByBA+Pkzct/FI3szI5y5xSIhhKfG2LMfSz7gh2TDF073DyAko3NWZNWFK6KVefq/fhxGL3p0bhReXK7IMBf/L5VzYYV6uzEgGFioGWtrhzEMiRKB3QaYC1k4AGY4GpR3jeR35meYJOwwlp5e5ApS4MeS7flmVEsZcZHGyKoyczKdyDpUtM3ILoitv8GrD1TT5m78SV/ardAU0WwzK3vW10Bzh4MqPDsyzFH0NWjel7D9sPbJ7OSYzSlcdx8uHK/+/tjcGqZVuyRETpYhRxdDq+R5mM71dux9+fXg+YduSzd+Cq9sS9FNn0ev5duAYa91PYAaUa0G2QmcCsEMM+ChF190ygzAs2Nm3HbUrlrkUn4pji4AbgEYRJqec7eLBcUwqP0nRfOHubP65QWJeCuvoDnb6gYKNOplhin7dA41uV10RwI4qdWUl0zJkit+O2rW8anYXBDSkyhG7mdVW2FfcziDit3uT5b3zFWCIVUItiuModOP4bGx20/xCwU1EouvUvBaSpR5kNU0K7LOnlCsgqdgLqDmfp6N3D3GDnQDGr108sJ1UnA/XHcCzVaQDIgMvrWO7W8yfAvQzvke6fontz82s8TlYy90mN4N926WbGnxPDgPLtOI4+TAdT0/FWIBAInhOK/A5w+/bt2L17938iDgBUr14dv/76Kzp37lzUpyMoZkTlCS4F1bXKwV4BF5Vd/i3Io/JW0ryeQMgBOBk78gtDXp+0Navg+cDZF2gyFTjyk/W2eqMpbKx5gWKJAZ2GK9KQAV2+BlTOXClvPJllSbmZ5sdROtMp4hbAbQs6m7t3stMYPtlvHruDtHiVTpnIs8yBqT00/1BPlZvtFXedhuJms5eB/d+yRMqAXk9xKjsNaDiOJVlxoZzMOvvQRWSg7XvM0mnxmlGoMaX791xBTw43PpabCRz4jsGsbkHAnk8p6tQaACTcBmr04fs6OpuTwMmH+JzbB1im5VkOGDqZLqGEm0BaLBBYi4LOCgux9s4hZvwMXc48oYgzbAnv5MUgZbmCn8XFNQyErjWIwoxhoiqT8d9SGUamuAaU2Emh4AE4ugMdPwF+32/dXck1kGVGJRGXALbt3va29bby7RgybufA0qmHQeUMqMqZP6bVUNy5uomiUFA9LsyYBjaXzisJ2/+d+XPvn2KZUMfPAd9KFK6dfHgt1h0J+FWlE86U6IvAytEs/1K55Yk/CnaqUqfQXTTtGMfeMRtZtlUSUToDbd5mXptpa3eNmqJ8+w/ZtS/2KrPFDEJPl2/pSjq3FAj7lwJXUF0AMjoyu34LbH+X3QFbv8VuZC2n83suKRxo/TYXNw78jwsaDcZSXJfq/pUWzd/HuSUs1W0whk5G0cVPIBA8BxT5t4tOp4O9vfUk3d7eHjrdA1peCp55YvKEHE+nguvw4O2szN+RE3ORq6GPE3RsSulmzCkJ3QQ0euHJjiV4PlA6Ac1f4kry4ZlcRVa5AU2n8kY4M8lcxDHl3GLm2xgmNZ5lmWmx7W3g/kk+FtIE6Pod4FmeP0edtV2CdXwec2yWDKTrw8CZhTyulNgEAHWG5B9C2mAcw5BttQ6+c4CfwdLBzLao2Ano9zsnCrf30bHkUYbvy8WXJUf7v6UzR6FkfoYhrFmKgz+wo9SVDfyv9Vt0//zR0Sh6hW7m6/Wbz8lg1Dk+fupPBkAH1gXsr3NC0/5DTixO/23+OvE3gNT7wK5PjM8H2C0nJ4MdtQxc2QD4VmOrc/dStj87wfOFb1VgzGZg6xucHMvkdKx1/ZoZUSURhR2D1vU64NCPLHOyd2TmTsWOwNoXKOD2/oWPPw5R54F/epqL2DmZdCBeWsOf645kqLoUKfc5FoefZNbOheXA8NV05RncI5ZkJrBs6OYu4MRcjhMZsSx7vbqRgsWw5RQXpAKYSwLqNCA3w1zEMeXIbJaeOnnxO8urHOBdGUi+A+x417jflQ0sX+3/O7DjQ6BSJ6D+aJb2BjemK8tQytb3N47Zpt97F1exO2jPH+hSNZAaBawcA9w/bnzs/FJmHHX+/Mnv6QQCgaCYU+RCTvv27fHqq69i2bJlCAri6mNERARef/11dOjQoahPR1DMiEpRw8PRHnaKgsvh9nJW/uf0kST6Em9AntSSq3JledWVjULIETw8Ln4M3Kw1gCudChVdF1lJnPx0+jyvzbfFzbROw31SlLSeO/sDpeoDQxYDmhyuaMoVXJVOiwbkMsAtGBj0F38+t8w8aLlCe2bBmIo4AI9zbinQ+Stg5wfm2wLrUBQ5/RfQ4WNmytg70oVSvg3Lm4IbAyn3mKtgi6xEPu/KeqD2IIohlbtSzFIo6c6p0Y+hpi7+wMi1QOJtfl7Jd42uOiky4ri6blidr9aLIc2WziVtDrDtTbqc1k7iY3o9SyRGradjaMRqvpdyrYGzi6zfU8xlIMck8Nzeib+TFSOtzyvuKoWi1m/l/a71zL0pqhbTguKHvSNQtgUwehOdW4a22Y+SuVQckdsDMVc4RqjceN1c3QRcWMl8GwcPBh/L7QFdLoVhy1IrW6RGAavHWV/P55cBQ5fSWZgYBrgF5h86HH2BGTp3DrF8y07Ja96WiAFQgPIqTzeR0pXB466BFB7ibwDH5tJFmBLOscCthLnpMmKApLu2t6uT876zHPj7azKN3zOG8dOU2KvMTWv7DvB3d5ad2Tnwb0GbQ1Ev8Q7HQqnFi9BNFO2rdOXPhi5vpiKOgfNLWQ5WTgg5AoHg2abIhZzZs2ejd+/eKFu2LEJCuMJ079491KxZE4sXLy7q0xEUM6JTswos6NiAl7MSkcn5lFZFX3h4W/eDCG7MyZk6teTffAuKDoWC+S8Ab4hDtwL7vuIkwqcyywaS77FjiAGZjGVJf3aijbzdBwzwTLnP0qizC7n6HdwIaPEKcHkDRZEafem+qT0YqDsM2J7X5alcK/Pjm3L6L2DMFmD0Rq5AZ2ewTEmnAdZO5HnUHU43gb0TcyM2vQZU6UGRVJOPkApwImfY59xSdsY5Po9lT96VgMH/sN3vtnc40avclVb8rFSWpKTkE4qpcmNI6bDlFKpys2y7dzLief4ymbFcTJubVzqwHSjdhJ27Gk9iB7CVo7gdoHvCyYe/PwNlWzI82vR3Bpkx1LVMc5a1nV3MiWilrnT8eJUX2TfPMy6+/O9Zwd6RrrQNL/FnR0+g31xmYe38MO9vvwsXQPZ8znGl46cUQR3cbR83LRpIui0dipubybFpzEaKvs7+PJY6xXw/uYLXesWOwJ3DHHNSIylSd/+eInBqpPTre5al++P8MjoGtTksqxq8CNj1Ecsym0wGLqzieFXScCuVvwBv78gxLz2a9z1bptNd03A8vw8M3y0GLq1mWamDJ7Ogku/xeXaO/C5q9Saw90vp1wL4vVK2JcXujDjg5Hzb+56YR0eqXcHeTwoEAkFxosjvFENCQnDmzBns3r0boaGhAIBq1aqhY8cn6BgkeGaITlHDsxCEnLP3kqU35mTyJq9qz4J5sZDGvIG4tYcOAoHgUcjNomiy53PjY7FXGTTZ9l2gSndmEQCceNzcldftJoYrwde3cTXz+g7j88OPAveO0da+/ztmFgxfRUeJRs2b7uhLXPnPj8jTwJlFFI3snYCtbwO9f6aIAzALp+ZAihtZSSxXbPU6sKAr0HgiULEzcHOn9XH9azCAVK8Hag7ghCc3k62GHT0AvYar2xXaAVW78XUur6NI1XQq3S6DF3FyKNUdpt5ITgBirnCF3nIiZ4k2F4AMgEnuT24WINNTQAqqS6HWTkWnzoWVQOUufJ5rAN0UR34BLq6gcJabybLLJpMp9uh1dFooXTj5i79ufJ2rG4Bbu4FJ+1maIRA8C9g7AI1fAC7nlTl1nwHs+tjY5QhgOdLN3XQUrhgJLOoL9P8DqNlfujQpPY77+9dgvpfKhePEoR+NYmpGHPc7Mhvo8CHLeY78wuuw0Qss8VGnAf7VgaO/ckw0EHEa2PYWg+C3v2v9+vaOfP6K0RSTDFxeB9zcQ/F5+QgAeqBUQwD5CCLFFJnSlXl/zr7SJbl1htE5ZpohlBhGca7BOH63nFpg3JabxbFv2DKKXj6V6WK8vR849huFvUF/s0X8xVXGsjgDjl4U1bLT+N3V9Tt+3qa/NwM5mfzugMT9ZGYiRfvMeH5vOPuIMiyBQFAieSpLfjKZDJ06dUKnTp2exssLijGRKWqEeD5Gi9N88HZRIjEjB9kaLVR2FjeEcVcB6LmyVhC4+DPP46YQcgSPQXoMsO9b6W2Hf2YewbWtDO5sNJElBQBDQ+8comNk54fWz9XrKeI0nMBJybUtnIgE1WNr6xr9mTPhVZ434lL41wCSwoCEG8bHrm1h+92tb7LE6tpWo5jS7gPenGenAsfnApMPAOtTgHsmVni/6ixjWDeJooxBCKo9GHAN4gr6lteNEzM7FUsVWlXkee94n4/v+5qTuc2vma+eV+/L93h8LgWUsL1c+Ve5GgUoU+xUnHRahs36VqFIe/B75j3893hVoM8cYO0ECsIAnTZ953EsuLSK2R9p0RTjDCKSvRNL5sq1NhdyADoX9s8Aes16vHbPAkFxxNmPIbdn/uF1YCriGMjNZBlprYH8//Z36FqTypHKTuFEf8OLxscC61L8WTuRJaIKJQWe8CMUoWsPZilWxfbsSLdsKMeNzERpMeD+KeZhNZoEnPrDOC44efO6jrlqLuL8d26pFCLavA3IlUBAdZaNlTj0wJY3gYELgPXTWFZqoEo3CjUL+0o/9cw/wPCV5kJOhQ78DP/qyu8klSudV64BxlD5u0e4mFG5K0Pu933D5wbWZSj+wt7sNApQ4GswDuj2Hd2aptQZwnHWkpQItqq/tcf4WFB9CkieZR7t4xEIBIKnTJEIOT///DMmTZoEBwcH/Pzzz/nu+8orrxTFKQmKKTEpatQJ9ijQY3rltTKPSclGaW+LL/aYKwBkgHvpgnvBwLq8SdDrRStMwaORESfdihzgJMfJG5hykJ1ezi+jgyXhFjsgxV3LPyw0/gZdOwCf71+DocgBtdhK26M0b5zXTeIqZYUOnAjdPwEENeQNcMVOzPRJDGOHkmO/AS+eAl48wcmZQVgCgMDawNFf+O+cDK6et36bq6R6HQAZ823WTeb2eiOBRf14c++a18Fp+TBj6RLAMMz93wHDV3AFNbAuP7PYqxR12n/IrIrsVIosYfuA9VOME7Cbu/l+233ISaIlzV+mw8aU2oOZlXFtm7mIAzAjY814oPWbxrKRzERgxXC2Ez/1Bz/LpYPNxaHcTIpfg/7hirJlLtGNHXntlYWQI3hGcPKii6LXbNvB6QDF1q7fUmhVutDFYYk6Ddj1Kdt8mxJ1Dtj9CdDqDY4Hrd4AbuSVNl5eC9QeQtE78Q6FgwbjWHZpKaaasvFlYPxOoHJnQK9lGVBGLDsl3T/BfexUdEg6efNY4cc49jR/heNiSgTH0pIWbJ6VxM/07hFgxMq8DJtEfgeEH+PPmXndBWVyivnuwSx1C9tHAdtQzqZ0YRbcxVXGstXsNI6DQ5cycD4ng89rPBmIPANU681xPC6UY/uyYRyLDei0wMk/KAYFN6TwBtDNWKal9ftRp9IFaSriAHytlaOZgfYslTQKBIJnniIRcmbOnIkRI0bAwcEBM2fOtLmfTCYTQs5zjDpXi+Ss3AJrPW7AkLkTmZJlLeTEXmUIon0+XXcelaC6LJFIuCnKIwSPhuIBf/sqV94Y+1almJJyjxk4VXvQKZNfUK5MbixRULlwUhUbSidPbiZvlq9tAV74lzk7l9bwBrzFa3yNjHgKQSnhFI5avEYnDHQUgTITeLNuQJtj/vO2d4CxW3lzf3UT28RGnmdpWKPJFKLSY4EmUxhQGn/DXMQB2Fa2y1dA1EWWizn7AgP+pAPu1J9cNZbbMUSz0+fAifnmGQ8q17wSsAHA0GV02MRfpxOp9VtsOb7zwzwxKYATPe9KFJXWTZH+XJPuAPbObNWbk5H33nOBi6vZYev0P9YOHwNn/mFmxPG5Fr9nF/6+BIJnBScvoO4IIPoyry9bNJlKMUBhT+dF2D4KJaZduzJigWubpZ8ffRHo8CmvPTsHlmwBFIF3fsDOdLs/BXrO5PhxdTMdOrbQ643X4qV1zOhq9AKQeJeicZ2hdN9e3Uw3YJkWFJBOLeAYfW4Z0OYdZn51/75kZbYYvi8CawOLB9ERevcIxTK9HhjUltvLtWEO241d/B7yqsDfo0LJ312NfkCLVznGH5tj/hp6PcXzar25OKFyY7ZNTjoQuoXOp5RIframIo4pJ+bTAZp8j+VeDcZKi2YZcfyOkyLqHLcLIUcgEJQgikTIuX37tuS/C4Jff/0VM2bMQHR0NOrUqYNffvkFjRs3trl/cnIyPvjgA6xduxaJiYkoU6YMZs2ahe7duxfoeQkendjUbAAF23ocYPtxANItyGMuF6wbB2BnHZkCuH1ACDmCh0OrZYcQlRsFhLRo6308SjMj4P5JYPVY46rmzT10mrR4jQHbhu5MllTsSNEGYMnR7k/oaIm5YtynTAuucJ5bYnzsxk6WFnX6HDjzN1/35h7uN+gfTsoMN9m1BlGEAbjyWn+MsdOVnYqhvi1eBfyqstSg1XSg6RS+74uruJ9rICCzYzmEKY6ebPFrKJswcH4ZV2trDeIxdBpOtlwDrLvUNBjHCV3SbQY295nNLIWMeODADD5efzTQ+QvulxrB49foa9spBbBlsaMnBaF6IwGXvO40wY2Ak7/bfl5iGCdBljSayFIUgaAkodfzmtNp2H3PclLs4gdU9AMcXMzHGAMN8hx9//QyPnZzN0s2u/+PxzR0kzKMf1LoNEBWMnD4R44ZBjEn5jKvuY6f0JFj6I5UvRfdjFLun9JN6ZBzCaTYHXWO7g9HD6DZi8zSWTrE/Hwd3JlDdnUTkHiTDsH4a4Amq0QJOXqlM2Q+lSlMp94HFvcHxm4CvCvQbZyZAFTrC9TsS7fMf2PkHpZHDV9FNxMA7P4MuLJO+oUSw4Cm0xgwXaYFc90cPfm5LehMsT6/wPzUSMCvBkvZrm8H/uoGjN3C8zQlOy3/vxuDu0ggEAhKCEW+5Pf5558jMzPT6vGsrCx8/vnnEs+wzYoVKzB9+nR88sknOHPmDOrUqYMuXbogNjZWcv+cnBx06tQJd+7cwerVq3Ht2jX8/vvvKFWqhNldn1Fi0vhFXdBdqxzsFXBWKaRbkMddLfi6aHtHCjhSLTQFAkvSooHDM4G5LYDlw4Ges6xLpJQuwOCFvHleP8X6ZjTmErdFngO6fmNd0udWik6Xc0vpRslMoPslM4mdYQA6UrwrSU+w4q5xZbxCB+NjORl0r4Tto0CzbjInMOXbcfu5pWxBXro5HS1Dl7E0YX4bYF4rCkkZ8cyOOfgjxSKAHaXKtjT+bKDeKODIz9ZlSACw92uWQAFcPe8xk0KT2fPzQkmXD2PgqaMHha0/OgDLhgARp3jsQzNZCpWTQbv95bV8fn5la27BDHSuN5Ith1eN4fsL20/hyha+Vc1zJwCgVCOWgMiFI0dQgkiPowvl93bAzBrAPz3Z6U0qgNyrAoVnU+QKCqb7LTLCOn1Bx8yi/sCsmgxPN+xvC1d/PsetFMeTOsNMXrs8RQHT72eDW8bymM6+LAc9tQAIP8xQZAA4/BPvG+QKdqeyRJ0C7HiXpUadvgTuHAECaktnthRn7Bz4uShUXCDQqOkwSr3PcXvHB0CbN5m7Zil0a3MpuieGUXhz8bf9Os3znPgn5vO74fBPRjeoXs8cpOBGtp/vUYbj95Y36ApKi2KYtmUOmso1f6ejEM8FAkEJo8jDjj/77DNMmTIFTk7mX2iZmZn47LPP8PHHHz/0sX788UdMnDgR48ZxFWfu3LnYsmULFixYgHffte4ysGDBAiQmJuLIkSOwt2cJQ9myZR//zQgKFINjpqCFHADwcVYhKsVitS0zkauHHgXsyAGYw3Frr8jJEeRPRhzLgQw1+5mJ7LoyeCHFk+hLXP2t1JnlBrcPSK8aAxRSJu/nSvTYrVyZTIvhhMajDHDvBDDwT5Ye7MobZyt3ZuBule5cDT2/zPa5XlrLEgHTdtpxoXQBqaqxjXb0JTp36o1gfsXpxexspc0Flg5iyZaBa1v5foYsYueXJpMZYHxpDUu3yrbiBMuwEhvS2Ji5Y4lex6ygbv9juLFOx+O5BrFEo0YfIPw48xgATu42v0Zxq+5wik6mpEZSuGn+Ko/tV4PBnkd/tX5t3yrMsslKpghkIOkOsGEqLf8Gt5ApMhkniXYqTpSyUxj27FeNbiKBoKSgTqUAYyqexoVSIO3zK1B7GKAwEUmcvChw1hzAUsfsFAq14cfNj1tnKN1ux34zPpYYxnLFGgPZGc6SkCbAvZMsvew7l+HG3pWBWoM5wddkG8VZA7cPUGQZvwMI3UpxNbAO4FWOTj2ZgqVTw1fRyZMWDRz8AWg8ybr800DEGUCrBq5sYmlS9fEPLp0tbqiTKdb3nEWh/NxSilqNXmDnqVv/8jtLykEK8PstN5PZZ2O3sF24ZZlp9T7MSjPt1Jh8lxlHLd+gU+fYnLx7tTLcZknTqQzHNuXaVp6baRmfXkc3quXvH6ATSJRVCQSCEkaRL/np9XrIJCa258+fh5fXA9rfmpCTk4PTp0+btS2Xy+Xo2LEjjh49KvmcjRs3olmzZnjxxRfh7++PmjVr4uuvv4ZWW/LaQj6LxKSqobKTw9E+n5W2x8TTWYnIZIsJcFwo/+9RCJ0K/GrQpptwq+CPLXh2SImwDl68dwJYMojBjd2+o9PDsPqrzbZ9LJ2GZUJx1ykONJ7CEN6QZhQlvMpT2CnVEGj/EdDsZaBaH3ar6vAJV8/zKx/SZEtPRBRKo0NIm8PV2W1vcyU+pD5wZSMnSqYijoGc9Lz23Z2BY78CQ5cATj58HwdmAAP+MAqtel3+tniZjKVi6TGAvYolaFkJnAyumcjuJ5bPP/kHJxJSZCZQgHH04LmXb8eyK7nJ+kdIE7Z1h8w6+8HAwe/ZUt0QNA1wIjtoIUWggJpA16854a3QTog4gpJHRhwzqqTY+SGQHmX9uJMXBQ7D337pptYlodX7AickShPPLgSq9aRzzdRhUb4d0ONHijDV+1EEyowDcjM4doU0ojihkRhHr2+nMJEWSbE8J5MCa/U+LNkZ/A+QfAcY+BdLxJLDeaz80OmAC8tZPqkvgQs6eh0Qc5GlaS1fB2oP5ed98g9g5RiKX3aqBx8jI45lZqM2mLf5dvEDWr3JsV6Ko78wYB+ge7P3T2xZbsDeid9xOelA9AXr1zUVjbKS+b1Uoy//rkz/biq0Z2aSk/cDPhCBQCAoXhSZI8fT0xMymQwymQyVK1c2E3O0Wi3S09MxZYqNMEkJ4uPjodVq4e9vbtf09/dHaGio5HPCwsLw77//YsSIEdi6dStu3ryJadOmITc3F5988onkc7Kzs5GdbfzST01NfehzFDwa0SlqeDsrJYW+J8XbWWldWhV7lZNjt0IorfOrBkDGWm+figV//OeQZ/JavHvY9rYr64EOH1NIMOBTmTegUuG5bkF08Wx5nT93/IxhnM7e7LySbfJ5VeoCtHsfmNuMYcDDV7BbTNv32EVJisqdrc/XNYA30+pUBgUbWvFmJrLLU51h3MfSjWLKnYNcdd3xPlfPu3zJ8OCI01ylbfUGb7DdStHpFn1R+jieZYF/enCFXK6ge8ijNFeLM+Kkn6PTsIRKrjAPRXZwz8v5sOPnn5vJSaejN106ALddWMVJSONJ0pNDANBkQ5uTAcXEfzkZgZ7vxzUw//KQYswzeS0KHp/EW7ZF1qwkTqLdgx98nIodgH/znBkyOQUVqbwvbS67xU0+zJwadSqv4/sngAVdGIh+9yhFFAN2DsCo9czfqtSJ44sld4+wtFHpyHFj6SDj+5LbAV2/o/g6fBXFA0dPCshS792znFEYjwsF7JwAzxCGohcghXot2jsCA/9mQPCcZmz/PXIt33NuFhcbMuLoerEsYwL4Xg2CybUtLJkdv5OdrzITKbSn3LftMtXm0K0lU7BcbsPL/H3716QL0sWfjq7YK7znir1qfG5wI47jBnLS+R135yDdlUOXcsy2U3Hx5PRCoOtX1ueg04kyV4FAUGwpMiFn1qxZ0Ov1GD9+PD777DO4uxsHWKVSibJly6JZs2aFeg46nQ5+fn6YP38+FAoFGjRogIiICMyYMcOmkPPNN9/gs88+K9TzEpDoVDU8C6GsCmC51tlwi1r9uFDAtVTh2J2VznRAhB8F6o8q+OM/hzyT16JjPi5EuZ25+wNgDX/rt9iC2xSZjCU8J+bzZ7dS/E/lDKweb32jfGMHc5zG76SDxdGbwb4yMDMn4oz5/g4eQJ3hLJUwfc1OX/B1YkOBTp8Bq8aai0zaXN6MO7jZfp8qN+P5he3lDXndkXSxHPgfsxgAOue6fMXXsJzc1RzAcgpDmYNOy9ycwQvpGMoPO5W5iAPws9TrOTbYObDkLOYSV+dv7WG2jkfpPEeATHoSY4JGpoTCLa+l+jPAM3ktCh4f0+50UjzoGjTgHsJr7MoGjiP5BQNrc4GcNApFy4YYr+Gg+hQJTEUcgCWaMjmgTgIC6lD4Tbpjvk9CGDNh7h03L/UBOOZsfYOi9619zMNKjQAaTrDO45LJOR5mxHN/pRv06mTILMfzAqBQr0WFI0WXCyv5c9R5LgI4eACbX2eZW4UOFM13fmj9/NZvGUueVO78nV1YAVTuwly4u0coqOR7DkqgySQ6O5XO/L2FH+W/g+rTOanN5feTVzkuCKRFA91n0PVlQCbnwkZ2Kkt2j88zf502JnEMWSnsznhmMd1k1frQMVbS2scLBIJnniITcsaMGQMAKFeuHJo3b/5fRs3j4uPjA4VCgZgY864kMTExCAiQtqYHBgbC3t4eCpNa7WrVqiE6Oho5OTlQKq1vGt577z1Mnz79v59TU1MREhJitZ/gyYlJVcOjgDtWGfB2USIxMxfqXC0cDKVbsaGAx0OsEj4uvlWB8GOFd/znjGfyWizd1NoNYqBab5YZmaJyYa5LUD1g//+MWQ4NxzNbJuo892s8Cbi+jVkztlY7z/yT133Fk6vTAbWBbe8CfedQtLi4mk6UCh2BFi8DsdcA/xps8epfk7kEdg5cjd/6BjvOjN/B7IjIM3ScqFzpEmowjpMzKWoPAS6t5r+DGzEbZ35rOnEG/MF8jORw3qQ7egFjNjGr5v4J2vQbjKcAtfUt62NfXMWWx94VgYSb1ttdA5ijE9KEkxKfysx/CD/OcoiFffkZu/ixrGpBF/Pw1mNzWFrl5G07v8GjNFJlbniW0heeyWtR8Pi4h3ByL1VqFFTPfEKdH87enIBX6sxA8pxMjiNpEqVZ3hUooN4+YD5+1hkqnWUFsG15xGmOM/3nA9d35IkUeqBqD5bx3NrDsc8WqVF0hyzsxa59NQfxeMd+Y1lWQG2OIWcXA6Gb+bmM2Qi4BT64DOkxKNRrUZPJ3wNAJ1LzV4CVoyjc9ZnNz/7KRiCoLjB8JUOK46/zd9PoBX7WN/K6VtUcwG6C+78FDs/i569OoejiVZ7jryVupei6cfIBLq9nK/K1ExkiXaoB8FdX475XN3I8H/QPvye8ypofy9mXotvhmZCkRj/+X50KnFtMQcjAlQ0UkMZsKpxMRYFAIHhMijzsuE0bY6tVtVqNnBzzTAY3t3xWbk1QKpVo0KAB9uzZg759+wKg42bPnj146aWXJJ/TokULLF26FDqdDvI8q+T169cRGBgoKeIAgEqlgkpV8F++AmuiU9SoE+JRKMf2dlb99xplffKszXFX2ZK5sPCrRjtxRgJvUAVPxDN5LdqpgH7zeXNq6mTxLAd0+pSOGkucvIAq3Sg+aLK50vx7O97YDviTLhLvynTVWE6A/GuwtbWTN+3iroHAvm+ZEdDhYwYmrxzNYzccD9g5UuzZ8BK7XNUZQldQYhi7Z7V9nyVXE3bx5l6bS/t8xY5cKd35IdDjB55zownASYscjfJtOSmKvphXuvAtW9zqNHQdtXmH7hy9lp9PXChFm3JtmCfjUxk4MpsOIynSomi77/YdsHoCV2Nr9ucKK8Cb8xu7GPhcKy+Mefs7DH8+8D9jR6mGEzgBsezAo9exo8qUQ0D/ecw2MnXnqFyBbjPgpCryr9pC5Zm8FgWPj2sgnRWL+5mXGDr7AP3m8f8Pi4s/3S5lWjD3pO8cYMVIlk4ZcHAHes/mQklapPnznbw59jQYyxIqnZbuukurgV2fcIzMzaAoO3pT3rnJmI+zdDDQ+Utp4Qig6O5dwdge/Z9ewJAlHL8G/MGynbhQYN0Uo6ilTub42eOHPDHH4eE/i4egUK9FvY6fe4vX6JRKiwJ6/cSw6WVDgfLtgWbT+D3hVorjdXYKEHEW2PmR8XOs0Z9jb3be/b1GzbyaDh8D+2cA3b5ljplp+a/SBRj0Nxcn/v2C++79imNw3RH8XVmSFk2RqNsMlkSZVq4q7OnsCfvXuOBhoOt3RrdkWrS5iGMg6Q6w7zs6tpT5dDAUCASCIqTI7y4zMzPx9ttvY+XKlUhIsG4j+yjBw9OnT8eYMWPQsGFDNG7cGLNmzUJGRsZ/XaxGjx6NUqVK4ZtvvgEATJ06FbNnz8arr76Kl19+GTdu3MDXX3+NV155pWDenOCx0ev1iE3LLpSOVQAdOQAQmZJFISczkbXd7oW4iuxXjf+/dxyo2r3wXkdQ8sjJoD188+sM6B2+kl1WMpOASh352IPKcAyr3CkRtLBnpwJbpnOVs94owNEd8Ktu3L9ab6467/2KDheAbp5273NyseUNdphKukvxQ6flSvPldQwOBugCMuXUAr6X3Z+wxGjUemuL/a1/+fwmU+m+ubyBN/I1+nESdXYxHUQNxlE4UacYnxtxmmKNrSDhzl/nv9of3JhlFHpwAufowTKptRN5DgolBZyKHYF1Jh1oKnQAVo8zHiegJsOSpdComc1wbindOXHXgIQbdAH5VQP2fg2H9p8CPsKxInhGUdhRSJ50AAjdxEmvfw1eAzkZxiySR8GrHEur4sM4rkScZpC7bxXAszRdEg4eHCtNHTSJtxmafvofBvLqNCzDaTCWYcjaXODsEoo7SXfoIjHtuhR/g+OiVDh7UD2O26asGgN0+Zqvu/l1Y4aZqdMy+gJkcgUXdUpSeY6dkuHO+7+jQAJQKGvxGgX+wz8Zw/pHb2SmTk4GULYFnaAyAGVa8vvk3y/pwjGQFk1BLum2cVv8dX6OnmU4dh7+id8Nkw4w42bP5/ybirlsXPiwdLRe30Hx//ZBoNXr5uHxbkH8ro0N5SKbkw/Dj10DjeW/17fZ/jwurgTavQcoC9HJLRAIBI9AkQs5b731Fvbu3YvffvsNo0aNwq+//oqIiAjMmzcP33777SMda8iQIYiLi8PHH3+M6Oho1K1bF9u3b/8vADk8PPw/5w0AhISEYMeOHXj99ddRu3ZtlCpVCq+++ireeeedAn2PgkcnJSsX2RodPAurtCrPkROVnBd4HHeN/y9Mm6yzH2967p8UQo7AnLhrwJKBnEwkh9OCH9yQN5QBtfMXcXRarnSmx/LfHmV4U7vPZPy8tIb5MJkJtK1nJQF1hwHLh5sHc0ad50rouG0sPVo+gtk5ZVryfHLS2Oll2HIe0zK0ODeTAg7A45qKMADfR7k2wO39wIoRnHjVG8mb5vXTgB7fc6VXq+HKtV7P8OTmL/P1s9MAz/Isfzr4vXnphtyON/vlWnK13SDCuAayw4pHaTpiHL1oldfkcDJg2mJdmwOcW8LJR5MpFGUqdgLsHdgWPTWKExidRLi0KTnpFKGWDaV45hZER1TsFQCAQmOjvE0geFZIjQDmt2FJsbMP3TIJN3mdTjkM+FV99GMasr4AILAecHAGcPovul7kdpyUa7JZNmMINFe5AAe+pzvGQE4Gy62aTgOq9OD1CrD7VfOXgR0fGPe9tBoYuIBdrHRaCjONJnBc0GkoPvT4ETg0k9e8TgOcXwp0+555OHJ7lmJmp3P8vH2AZVdaTf5d94ojWi2wfqp5yWhmArDrI3YaC6ht7BalTua9TnYaxZkKHSjGZSQwpH7wQrYENzt+XtZZ9EVg40vAoEVAtV7MFrJTsZRr10cUygxCoELJ30v7jyiw52TQvRN1Djj8M3+3HiHMJsxOBzKv8PvGJYCCo2sA/6vQ1ngeOZkU9XIy8+9Eps3hgkliGO/vVA/IhhIIBIJCpsiFnE2bNmHhwoVo27Ytxo0bh1atWqFixYooU6YMlixZghEjRjzS8V566SWbpVT79u2zeqxZs2Y4dkzklhQ3YlJpxy4sR47STg43BztEpeRNqOJC2QmhMDpWGZDJ6CYwuBkEAoA1+P9+aX5Tr9excwZAF0lz6TENuWquCK8eZyzz6TcfOPSjxX6ZwPG5LAnq9ROFo5N/SE8kslN5g126GY/t6MlSq9XjGYQM5NnSp9DebhoCWqGD8bwB3oAbQkQ9SrOl69nFPF7EGd4kH53Nfb3yBBp1EnB0DldCR6zmCvCxOeaBzsGNgKGLgRWj+L69K9LinpnM62vAAmDvl7wR7zUL2P4eV3cBXoc1BrBN7bzW0p/r1Q3A5EPAkZ8pxhhWe73Kc9KWncauO1Kr9ABdP4bfR+yV/wSc/17/cSaxAkFJQZsLnPqLQmjUOfNtOg0n2D1+eLKSFDt7fp/GhRqPe2AGW4MPXEDH3L3jHH9MRRxTTv/FDJ1qvZmpcu8Eyzs7fgYc+YlO3ZwMIPIsMGINsPVNoNV0IGy/9bjQcybLgxRKlpiumWAUPOQKhu+WakCH48C/oFe5QqZyffz3/zRIui2d+wXwsx/wB/BnJ7ZpV6gorlXvQzHl+DwuUBjwqwZ0/ooCWWok3Tv2eWVmrgHsBLbhRfM24pU6c/y9uQco14oifUYsULUbsOk1llwZKNOCfwcHZlA82/URS2ibTuXz/atzH8NrGshMYIv7g9+zRKz3z7Y/j9LNgEtrmQPXcDxLyVz8HukjFQgEgoKkyIWcxMRElC9fHgDzcBITEwEALVu2xNSpU4v6dATFhOhUOmUKy5EDAN4uKkSYOnLcggqnY5UpvlUYpqjTlthWw4ICJicdiDpre/vt/UDjidKlCCn36OTxLMPVZWdf/h1nJlrve3M3BZVeP9MVc/IPoMWrzN/JiGXbVkPA5L1jFEfCj7JMa+Vo85BkbS5DL7vPMDpOqvbkROXg97yB12n4Gm3fY35Ouw/Y9jwrmY6fqIu8GVc60wlUvi1f7/4Jo509M4GlXLf3m7+X+yeBXZ8CY7cACbfoSNr1MdD6ba6g6zT8zMq0oOvIVHAx5GO4+PGcL6+1/qz0eh7z4kpzsSsxDNgwjZ9h2/e4amwphjUYB2TG86Z+k0SZbpO835NAUJzITuN/crsnn4zmZAD3jtreHnGC7pQnzRYp3YxijkGkDT/KCX3L6UD/P+jOib1s+/m5WRSbfauw9CvmMstFy7VmPpedI8fBXDWQeg/o9DmD2y+sMD+OYVzo+i0dOOsmm+dn6bTA2UUUKNQpHM9KNWTL7JLE/VO2tyXd4fdCpS5AmeYUZsL28d5KoTQXcQCWn255g6LXxpf4PeMWDEzcS0fNylFGkc7AjZ10VbZ4jU6odh9wnN48naKdKXcP83W7fJOXOacHrqxn6W2lrhR2BiywFtVv7TWWzWpzefyyrazFQIU90OIVdlHU6/hd514aaPYSoBD3dgKB4Okgf/AuBUv58uVx+/ZtAEDVqlWxciXbGm7atAkeHh5FfTqCYkJMnpDj4VR4woqXsxJRySaOHPciqHP2qUp3hOkKveD5RqHM3wnmVZGTA0t0Orpb2n/Im8erm4A9n9Hubau9b8JNij8JN4AOn9AVs+czIHQLRZ32eXk27iF0s1TsBERfst3p6sJKhpf6VWcY8JKBdPAMX8mV8NgrvPkeu40rtL5VgBf+5TVQsQNXxP/9ksKTVwUKO6bCioO77eDiiFNc8U+8RVdP0m0gPZqfZWIYS8virtl2zZz+i6vFttDrpB1L6bE834QwYOQ6hjkrXViC1uUbdh6LvczQ5sGLONlUugC+VaDv+xtzGvJrvy4QFCW5al7jaycBc1sA//QEzi3j3/njYqdiiact3IIpkjwp7qWYl9PqLYpPDu4cs8q3YUbVkoH5P18m47WZfA9o/zHQ7EWGK0edB65tZ4aWOoX7yRR5zsb50sdKjwVkdhSfLUPQDZz6kwHxZxdBlp0CpD3BZ/wUkHmVt73RwZ3OmnYf8HexchTHwfqjWGomRdJt7jt8JbONVo+j0ykl3FrEMXBlPX8PWg2/dyp1ss4pMhC2l98RSbeNj51bCkAH1B5K8SX6MqDOC6RPjwH2fW1+jN2fAg3HUZg3lOdW7gIMXcYSOdPr5PBMfgcJBALBU6LIHTnjxo3D+fPn0aZNG7z77rvo1asXZs+ejdzcXPz4448PPoDgmSQmRQ13R3vYKwpPW/R2VuJ2fF7ni7irQNk2+T+hQF60Im8I759inbhA4OxjdL1YIpMBDUazo5Ql2mw+N+UebzYNXFzJ1q6m2S8GHD0BvxpcwV4z3vh4VhJXFpu9CNQdTpu4JoftYU//bfvcm7/CcE/TVrGn/waubWN3maNzaId3L8VjdZvBidGBGQw9NnB1IwWbsdt4jhnxfFyTnX+OhDqVAlCHT3mTfX0bzylsL/OoUiNsPzc3i6vGlsjtGKjp6MkuKfdP8ubfdHKWEceV3GNzmIUReYYTuHOLuaoPsGTDsxzQ6QsAeiD2KiK8W8BX5QXR30lQbIg6D/zdnS42gCLl+ilAnWHMInnYVuGm2DtSXL60Rnp7qzcAhwIqK3IvBbR9l7k10NNFsWIUEJ3XiUidTGHa0HHOlEpdOE6E7ef21AhO2H0qUyheOphjwOV17Fwnl5t3UrJEqzafyMvt6Pqr2p3i+r3jgKM3x7XcTEBVwgTd0k34u5US9uuNBC4sB9p9yHuc7j/ws9LmcnHBFpmJvC9KjeDf3JUNxvB9KXyqACp3/i4c3el2avQCXVKmHQINZFm4UzVqPsctkN9BJ3+n46ZGX4pDibct9s9mWXFIE6Dr//i8c0v4mOXfQlaSeZc2gUAgKGKK3JHz+uuv/9clqmPHjggNDcXSpUtx9uxZvPrqq0V9OoJiQnSqGp6F6MYBAB8XFaJS1JxYpkUDHkXgyLF3YOBffhZlwfNHmZYUIAxBwQBXtQcssB3ArVDR7XHid/PHL60BqvUEylrkvzh5A31/4+rkjnelj3l8LtB4Cm9GN74ExF9jqKgUAbVppzcVcQykxwB3DjN7IjORK7X7vwN+b88bdlMRx4Amm92u6o/lz3IFS8Zk+Xwt6XKZi6DJpLgScYY36i2n83XzCy9XulBo8a1ifMzJi7k8eh2wqB9v1mND2UElpIlxPxc/YPu7zF5Iucuyil0fGUUcgKVqwQ3pGMqIR7JPPfx2KgVxaeJGX1BMyIgDtrxuFHFMOb/MmIn1OHhXBHrMpJhhQK6g6BpQ8/GPK4XCjv/dPUZR1SDiAGxn3WuWtesxqB7Q8VMGIcvlQNMp/By2TGc5pn9NisSHZlGM9ggBPMpS4LWFgwefB3C8HbaM7rvNrzMzJ/4Gy1BrDwPsndnFqQSh1+QCw1bQfWNK5S5AUH3g/mlm6Jz5h8G/rgEsH5MSzA04eXIcXTWGbsym0/i5S9H2PQqEGbH8PpnbEvitOUPoB/3N7yRTZDLrkmSlM+/BNDmAb2UKeNve5D2gQklnpRT3jgMZMRSxTsyXFvScfQF7IdMLBIKnR5E7ciwpU6YMypShJXf16tUYOPAB1ljBM0lMqrpQ83EAwMdFifRsDVLvh8INyN8KXqAvXNm6nlvwfOPsTVdOg7FAzCW2//atSpu/ZRijAbmcN5+GwE0D2lxgzQtcTW//Ie3tPlUAF1927dBk2y6b0Gkp3pyYD7R8DVg3hXk2Klfr1c4yzRhUaYurG4FWbwI+FfNW/LXMTrh7xPZz7hzkivjB79lpKvIsV7SvbrTet0xzugnirzM8tf1HFHB2vsfJ1IDfeWPtVV5abKo/im6brt/xpjw1goLamgnGzA2A7XTvHgKGLKH1X+XGyamTF0NRYy4bM4EsSboDpEYivspQzDypRlauDtmaB3S8EgiKCnWKufhoyd3DDKV9HBzc6Eqr2J55KDoNr0tn34Lv7qNOAw7O5ETb0hGRfBfY8ibQ/gN2wMvJoBCRch9YNoRi+em/gCOzmfmlTqErw05FATcjgePzn50ZaNz8FZajWuJWipN8jxAKvV2/pdibcMu4z83d/Ewn7IJe6QKZc4D1cYozUeeBUvXptlSn8rPyKkcBff0UoPmrHG9VbgwqHvAHcPcoUG8McGKu9fF8KlOE8anIn6MvAusmAeO2s+V7lIkgV603v9sc8o5t6rC6toW5Q0MWAUuHGB1AlTpzQcGAQUgM20dh3j0YOPUHGwqok5mTNGABw4sTw8yDulWubFcvt+P3cnoMhaLSzdixKjEMqDsCcAksmM9aIBAIHoMideRoNBpcunQJ169fN3t8w4YNqFOnziN3rBI8O0SnqOFRyEKOt0teC/J7twDICrdjlSm+VZhRkpVcNK8nKBk4uAHeFZjbUrkL3ShSHTUizgCb3wA2vmq+2m1KbhY7qDh68Gb17ELenCbdebiAzZjLXLF0D+ZkZOQaulcMKJ15k2xLZAIoRvnXYlcrnZaPaXPzX51V2HOSVWsIO1NteweoPZgtaE3dSuXbMXA4dDMwZDEFsMvrKLp0+pKvt2QQu1UNW87JhwG5HVB/NCeVG1/mTfm+7/j8iFPmIo4BTTbzLZq/ykmMoydXjsP2UkwbuhSoPcTqaXo7J+jrjkSORo9mpZ2RmJ4DJ3sRhCkoJsgU5teVJU+aY6N0YlZWlW68hr3KFU6L5ow44HheS297ifEt6TawfhrzuADmAWmyOak/9SfHiwptKYqrU4E7h+jm0+tYwnrydwaYrxzFkOJmL5s3RgioBfT+hd0CE2+z41JWsrmIYyA3i6WlcnnJC8V1L8XMMxd/4MxC4PBPwPIRfN/1x9BtFLoFCGrALoKpUXQpVe/Fkl3TBg/BDemiSY0yfj8Y2PQK0GcOxXoDdYZSgDHkvFmSk87Mtuq9+TddtRfQZCoXJQB2rBqzGUiLpCP6/ik6htyDKS5mp/F77+om7l+zH7N7/GtSnBq7hSV67qWAsZsZaj98Fb+nZHJm7lTunP/1JBAIBIVMkTlyLl26hJ49e+LePQ7Iffr0wW+//YbBgwfj0qVLmDhxIrZs2VJUpyMoZsSkZaNKQOG25vTOa20eGRWBKq4B0l2BCgPfvC4JEaeAih2L5jUFJZ/MRJYBHJtjfKxyJ05ccjOt9+87j+U+103CgnVaoNFEiolx16yfo3RhSUBuJl0oQfV4g6vNYRmCnQNFHGcfQC9nS92be6TPt9ZAQC4zd59FnqXzCDOln1OtN2+8vcpyJVybw/KmRi8wXNIQ5Hz/JFfUe89mjkVqpPEYR35i6LBGzVIGWV7mjYNH3vPtmeFj6DgVfx1o+w4DWI/9Kn1eAO3/HT/jcxb3N8/fOTEf6PIVJ4dX1vMxuR1kzl7APz0Q1Hs2fJy1cGhcDf5u+YhfAkFR4uQFVOhAp4glMjlddwWETqeHXF5Ik9zkuxRdwvYBg/6y7iploEp34OIqoMPHdMbs/sS47fwyoHx7oMlkjlGp99lVqfVb7GQE0PEXdxWoNYAigy6X40n8DTpS0mMpcqhceS62CNvHMPmShrM3Fxh2fUxxzqcSxXm5HQWQLdMpbJRuSvditd50ZapT+L1TcwAdUQol3Td/9+DYbFmmFH+dAdSt32KZVWYix/OAmsBtG63kAY7RgxcBdUdSpIm5wr8He2eKfX/3MDpYL6+lgDPwL5Z1nV3Esq1SDYCYi8CBHyjKjFpPAcfFpNOgRznmHi0banRiXl4LHJwBjN3K1uYFgU4nnY8nEAgENigyIeedd95BxYoVMXv2bCxbtgzLli3D1atXMWHCBGzfvh2OjgXQ0UBQItFodUhIzy700ipPJyXkMiAiPqloOlYZcA2i9fi+EHIEj0BimLmIAzAfp+s3zGAwLbEKrA3oteYijkxGd0tcKMuuVo7hKuZ/2+UsBzBk0oQ0YUnWtS3AiT+AoLr8e42+TPdQRiyt6FW6URgxJaQJb371WnOhSafhim7L14FDFmKOezCt6StG8Aa781d8XJvD92353qt04wq7qYgDUGg5/hswbCW7mfz7OYWnSp14npYh0EonlnI5eACu+djiVa78jPd8IR2ivPMjYPgKo5DT7X/AmUU8v+NzofSpjLaNqhTeZFYgeFQc3Fla+FdXTnRN6fHDE7chT1fnIjJZjVWn7yEyOQudqgegcTkvBHkU8P2d0pn/VyfTdVh3BANpTfGrTmHq0hqKzAbnhSlh/3JccfZj2+k2b3MyrXIFag6kW/LCcpZhb3yJDklLNGqWj1rmyJiicuN4nJXCwN6SgsyOzmVtLsUcKVRuxjDkqxvpxFG5cSw8vcB6//3fMXdHYc/jAnRYuvgCl1bTsVOhPUuY5Er+LmyhcgVu76MLtPmr/Pvd+QGzijZMsy5Dzohjq/FR6+ikykmno8jBHej9MwW6DdOA0RalvenRXGCwLKfNSmJp2JAlFLweB62GjqPQzVywCKpPl5F7aeZACQQCQT4U2Shx8uRJ7Ny5E3Xr1kWrVq2wbNkyvP/++xg1alRRnYKgmBKfngOdHvB0LlwhRy6XwcdFhYjkbCAon1DUgkYmoyPi3omie01ByUanA079Zf142D7edI5YzQDhhBvMeanWmzeUpvjVoCPGvwYFkEF/U+iIuUTRpXIXdpyq0Q+o3I3CxsLexpvfK+uZIzFhB7Dvf0ClDsDyUUDHj4HqfSmS6DS0wPtWAxb1ZQeT2kOYQWHg5B9c+Ry3nZOtnEygVD2WYGx8iZOAip1YZmbnwImRJeVa84b3xk7rbSpXhqwu6svJmOnrdvqC7+/yOj7m6Enx5t5JOvKGr7TdpavBOCAjkZ2xpNDreOPf5m3efDt6coUaoOOh3ijYJd4EvIooi0sgeBh8KgIT/wVu7OL15B7Mv3WP0nToPSYZ2RpsuhCF99Ze/O+xLRejEejugBWTm6G010OUeD4sbkF0F2Um0kXTezZF67OL6QCpNZBj07qpdIVcWmf7WOeXsnTz/DJ+Jv416MTRqIHf2+Vd59c5rhnEZYU9P6vsNI5fZxYBnb+gy0OKhuMhg55OlZIk5KhT6L6pM4wiA8Bx084RyE6hiF6zPwPoDZz8HWjxGgUwW9w5QCfP3SM8tkdp4I8Oxu1X1vOxvnNZymoQyy2pP5pCjL0jf48j1tAhlRZtu5vU3cP8nR6dA9w3cY+eWwrUGsT/MhPMRc2kO9IdsgA6jWKvchEjv3bttog6B/zT0yiGXdmQJzatp9NJlG4JBIJ8KDIhJz4+HkFBQQAAd3d3ODs7o2nTpkX18oJiTHQqJ26F7cgBAG8nO0TGqbjaUZT4VOFqlbDOCh4KHVebpbiygTfA045x5dNOyfatluVWdipOanRaCjpLBnKV06scb0yXDeNkp3JXihF/dbVewQyqS0Gpajdg4yuAJos5NM6+QPk2fI2Iswyx7DmT7p7mL7O8KvaK8Thh+zgRqjUQSI/j6uv1Hbzh9q3KvAVNNtDnV2DdZPOVT9dATgwy463PD+BE4MQ8cxHHwO5P6Jq5vC4vzPR34PJ6vl61HmwN3Pxl4Mgv5s8r1QAo15Y351KvaSAngxkbK0cz6NOAXsf/TB1QAkFxwaM023fXG80ckwL4TopLy8b76y5aPR6Vosa3265ixsA6cFYV0C2nayAwZCnFW42aLgpHL6DldOhr9oMsO40txtXJvO5zM2wfy8GdpTU3dvHnmMssIfqzk/Hav/Uv0DCvC1XVHnT8ZSbmhdM7AlW6UuhoMoWdAE0JacxW19kZgEPh3+MUKNpsupacfVkqFdKYwk5OOkuZ0mKYDZRm0oI9JwOQ2/O7whY5GRSDZHIKJ4v7W++THM4Fgep9KNicWWi+vVwbLkao3ICG41m+pdfT3ZOTz+/b8PqmIo6Bi6uAAX8gLSsHMzZcQoMynmhc1hOBUqXMpmiz+b3Y/3e6Vx+W1CiWeVm2d9eo+fikfRQtBQKBwAZFJuTIZDKkpaXBwcEBer0eMpkMWVlZSE01r5V1c3uEQVDwTBCdQiHHq5AdOQDgrcxFhN4H8CjiVTG/qsC5xewQ9LhdQQTPD3I7Ch+hm6W3V+zEEiE7JVcDD3zPvIeYS7yZBVhS1eI13lAbVq/Dj/I/AzJ5XkbAZekVzGq9gP3/Y3iwaa5BRhxwcTVvoB3dgXmtjM938Wcgcco9ToD8agCBtYBFfXgOBmoPpTMg9iq39f2NJUwTdgPXNnNyEFiHx9v8GidJUlk/5VpTSJFCrwMSwoAePwJlW9JF5ORJS/zhX/KuRT0DksP2UQwr04I31itHAUOXsETDVJQyxb86sHoCb7ZNyy58q/K9VOos/TyBoDhgZ//gfR6So7cS/ht6LNlxOQbvdcspOCFHrmAI8bTjwM1dHL9KNwXKteL1vmo0XYbDV/JnZ18g/Jj0seqPpYvEgLMvA+ZNx0O9Htj1KTvjrR5n3hVvymEK2StGMBB92LK8sSQLKNuKTo6IsxzLZCVsEcfJm04VJx+g3QfAmvHm41ydYRxXTanQngsHZVuzs5QUlbsyh6jOMNtjK0AnTvNX6TitOYAOMk02RZzUCObUXNkERF+goDPgD6BiZ34v2sKjtHSprIGrmxHX+FMsPHoFC4/ehbezEkcnV4BSJoPkH7izL3/XN3bws3kUISczntlvUqTH8HtWCDkCgSAfiuxbRa/Xo3LlyvD09ISXlxfS09NRr149eHp6wtPTEx4eHvD09Cyq0xEUI2LT1LCTy+DqUPi6oo88A/f1PkWbkQPwRk8mF23IBQ9PcEOWLFmidAFavcEsmpQIYOeHXJms0hUYtJDiQ5VuXHVMvstJRJt3pY8zeCHFDmdfTnraf8gbYgMKJQM+pcqdHNx5077rY/NJT3oMV7OVrmx/7l8DWDLYXMQBaL2/to3XRb95PI8yzYHkO+z+kh4LHPuNIk1yOANNu35nPRkyuF9skZ1K8UevYwcSyNi5RmFP4eb4PL5G7BXm2+z8kMJRZjz/a/O29ASsUiceV5vNfU7mOXJkcrZUz0594swRgaCkkJ6tsblNq9NDZ0vleVzs7BmS3ngi0GsWSzy1uZAtHcSx5twSYPlwIHQTy3ikvvOdfSjmmjou7FTSYfLNX7QWcQBe54Ycl2NzgFVjgdhQjiU73mO5ZZ4IrtflFsQ7Lzpkcn6PtHodWD7MOiPo/DJ2CCvbij+7B7Pr08EZQLMX+f1hSXBDfr5Jt7lQkR5j+/W1uUBSGJ2VJ35nWW7SHXa52v4u29AbRJvsVP6+Vc4Mrq41SPqYnb+kOGWL3Aycjs7F/H7BODTGF5v6KZGllUHfZJr0/m3e4feJXk8H56OgfcDfgybn0Y4nEAieO4rMkbN3796ieilBCSM6RQ0vZyXkRVAL7KOPRyz8oVE4Ft0fP8BJt2c55uQ0GFuUrywoqbgFsQ34qT+Z45KbBVTpwY5LXnmtwTNi6dzZ/o5RKLF3BFq/TSdJ4m2WDkVfYLeOo7/klQ5UpgNmx/vs/GGgdDNg4J90mWSnslwguAlXZOUK87ax1ftYBwmbcvovtumNOi89MQIofvT6mZ2oAE4I2n3AziaX17LTm9IZqNQVqD+SYsyw5SyFun+SQonKjWJRzGXp1/CvwTKLvV9x9X3fN3w86hxv9s8sZMCy6ecAABXasWNK0h12Jtn3DYVYZ19a/T3LQX/6H2DUeshCt1B8KtuKn7dMwc5b+QWgCgTPEM0qeNvcViPIrUgWanDnkHmZik7DTJR1k9jeOnQLy2f0Wranbv4ScGEVM3Iiz/I5qZEsrTJFoWTplqWIA9B9U6UbnYPpMRS1TZ0oMhkduXot9ApnlKzEEz3LUBNu2R7DT/3FroF+1bigsPFlfhcd/gkYupQd/u4c5DhdZxiFnLWTeDylC9B3jvRxAbpFMxI4tkuN78GNgFMmgco6DUtno87R9VO6KXPe0qJYJtz0RTpg8nG56GoOQgdPLbzWjaHY5BrAvx3/mmw0cPJPuk39awJt3+ff1L0TdGGqHsGNA1BIVDpLl4LZOYiFAIFA8ECKbC7bpk2bonopQQkjOlVdJPk4AOCdGwkdAhGdoUewaxHfUvlWNS9rEQgehHspoO17FAX0egYSq5zN91k/lTewIY0pMChdOXlwDwHOLctztsyhiNHlG+6r07LrkqV4EX6U3aWav0zh48JKYOxm4PRCnsPxecZ9XfyAyHO2zz05nO1/o8/b3iczgZMqBw+g/ih2v8pKoqW/2cvsRpOVxK4xCzpzMtX+I97g1xvJMM5Tf7Jt7erx1s6cyl04CWk4AchJ48QhoC7FmYx4TvpKNQAiTps/z86BpVxrJnICEH8d+p6zEJepwd1kDbJ1CgS7yJDRejb8ff3h41uNgc4yOWCvYhccgeAZQKPVISYtG6lZuXCwl8PLWQV3R+uSrCAPB3Sq7oddV2LNHlfIZfi8T014OasK/2Qtyy4NpEYyI2zMZpY42akoAqweD0SeYY6WayAn/Hodx8XaQ4xtzVUudOdJceYflhF1+RpYM8F6e/0xgJ0D9JpsqJ3dUYCRz4WPVstx9uivtvfJTACC6gAeIcC+b41dBe8eZqlv3RFAp88pVuz5HDg8y/jcnHSKZOXbM4vHFLkdx3WFPf9t2TGqfFs6Ti1zyOKvcTFj82t0rvb4gSVK8TeArW8CWYl0n4ZuNc+hc/AAmr8CeWAdeOVmAAP+pHMnsDaflxjGRYEGYynmJ90GwvYCWjXH/cfp+uYSAHT8jMe3pP1HQsgRCAQPRPS2Ezx1olPU8HAquFr9/PDNDAPQAJHpOgS7FnG9ul91tnY2hL0KBA+Dwl56BVGnAy6t5Q1uqzeZ/bLnC64KK5Ts1tRkMqBQMfPBNCNi+ArehEpx9wjQ4nVmxbR9h217q/VkqK93JbpoksMpLAXWodtHiqB6XKH0q2H7vXmWZYbEwAXA0dn8T6/ninjXb5mT41YKOPobO5Ic+pHnU7M/u45EnuZKePW+wISdwN6vja6ZeiOBUo3oilk/lfvaO7E9bv/fuSq88wN2Rok4zUlbdhonCA3H0wXVdAo/B2d/4PTf2CLvh6UXUjGyaRnYu7pi8Ym7+KJPEODk/3C/S4GgBJGUkYPNFyIxY+c1pGZxIt2yog++7l8Tpb3MBWUvZxW+6lsLrSpFY/6BMCRm5KBxWS+81bUKKvg+fjesRyKkse1tHqU5Vm17mxkr3hUo4gDAljeBPrMZJH91E3B2CdB/Pks9j/zM72wPG93n0qKBwzOBzl+xbfXeL4HoSywzajKFr3XsN6DpFNgrSpYfBzIZ3S1lmjNQXgrPsnTNLO4HDF7EEPxTCyjAl2lOMV+hAnJiOa7mZtJd4xFCgT0xDKg3HCjfms/LTMj77nmPTpfwo3RhHp/Lfzt5M3jZryqwfIT1+ZRuziB9ADj4A78LXQOA69s5vvvXpGtq5Bp+l4VuAbzLA91/APZ+A+z9wvgd1PEzwKsihX+A573zQ+NrKZQUhRpP4kLdo2KnZAMAz3Ls/BV/nX+X7T7gooa946MfUyAQPFfI9PqCLlx+tklNTYW7uztSUlJEMHMB0eGHfajk74oxzcoW7gtps6FeOBTjNO9gZjsH9KtcxB0kMuJYYz94EVC9d9G+9jPIc38t5mYxYDMnA6ja0/wG00BwI6D3z8Cpv81vxAcvtB0QDHBCcns/kKvmjXfiba44y+VcfdTrmU+gdADmt7VeLZXbAS/sYYmRXgMsHSKdhdB9Bju6rZ1ovV0mB4YsAiLPs+tM/HWg3hi2vfWrnle2pOckQZvLf2fE80ZfrwXsnYG7R4FNL1oHOVftwfKyQzP5c9lWLLMq1YDhy46ezL1w8TMGH2clIXLobmwM0+PwrXgMbRSCYE8n1AnxsP05Pic899fiM4her8e6sxGYvtLaUVfaywkrJzdDgLuD5PPi0rKh1evhrLSDm4R7p9BIuc+W4emxFFLqj+Z1rk5h4PutvQxuD6oPxN9kKLIBuR3H0Zavs6X0rT2cwNccQMeNvSMdIJFnWI5pGlLbeBKD5bOS6d7R5nBMur6D4vOIVYCDJ+AWDDhYOCoLmIK8FvUxlyFbNhQYsoSCVvm2LAWKOs9W6xnxzDdLj6Hrc/BiiibabCAni90CQxrxs3UNZLlVg7F0Ruq1fF5uJhcZ3EtTBEq5R3HH3gUo3woIPw4o7Ph5yhQs+b28Dmj3Pr+XHNyBhJv8naiTgXHb+b115wDdUolhFEfqjqBg4l4KWPMC/0ZGb+DvzLsCW4Cbdt8CKGSNXAvc2sfg5QZj2GI8M5FlxfdPAlMOsYT4SdBqGMCs19EN6hb4ZMcTCATPDcKRI3jqxKRmo0k52/X1BUbKfTjIcuBmr0NE+lPQL519aaUNPyqEHMGTo1ABAXV4E7rnM+l97p8EksKB4AaAdiyt4toclhbkhyaLq5lyBUUdOwdgzyfsQqVyZ1eq8KO0zvefD+z+lC4dgCVdHT/la2enA3FXuPq56RU6gwDefLd9nzew0RekRR69Djg2l2JU5a6095drw240YXuBs4sZznx1M3BpNQOZK3biDf7/2Tvv8CjK9Q3f2ze72fTe6L33XhVBERBBiiLYezn2ctDfOfZy7GJXrIgiiKDSpEmV3ksgIaT3Xrbv7483IVmyoShV576uXJCZ2dnZze7MN8/3vs+z7h3ofx8svMd3GteBX2DKTbVCTlWhDPKtJbDtc/HaMIXC5O9h62dQcBhPQl9+OVhMUrGGp65sS5XdSasoy8n/TgoKFyG5pTZeXeK7VSm1sJLDeeU+hRyVSkVEQP3l54TAOPGz2vkdRLUT8WHVi9Lq2PtuaD9eYp2riqSSoi5up9ysd5oo5yqPWwSIfvfDkVVy/qnIheguMOxZSSna+6NUmXS/Rc5he36Qc6y1RPxZ+t4r/276CE9MV1QdJwBnV8g5o6i11UbOHqm8WTpdXlujvjBmBtitUJEj7atTfpT0xAMLRYRvOxb63w/pmyCstVxLLJGw8H7xmtm/sPq8bYPml0or25aZYA6VFqbQFuJftPol38e29QvZ38b3ILKDXHMC4wEVfDpUxP6Bj4jo9McHUpEz6HExoy7NFGHOLxgKUyQN63gRB2TCYs1rcMX/qj9P70pUvSUGut8obXOn64tzPKVZUv256cPa9/ay5yToQHeevkcKCgoXDYqQo3BeKbc5Kbc5z0n0eM2NZrifmvSyE6TcnE0i2shNooLCX0WtFsPhoqMyu9gQRUfA6RDxZvxnIsoExotB58FF9bdvMqjWM0ZrlNnTmSNqTUT7PwC/vyImnyAx5/0fFONGc4TMWteIQN1vhJ53QNpGGVSrNOJFUJYlM+X7F0Le/oaPvSRNBB9zKEyaDcGNpP1q60y45nNpk6hrQJq4SCqJbl4qlUq+0rZq8Ljhuh9k9rggGebfIVVz0xbA0qelnWznt5J+A9gGPEGCLZpdeVnszijG44ZO8UrSosLfkyqni6yShr8/O9OK6d887KT7cbhcVNhcGHRq/HR/bchZbnXidLsJ9NOhaigcIawFxHeXKsBjD8yB354WQbjHLSLu5O6T9s8ak+MaklaKEfK+H2Hgw7DnR9j/U+36zG1SWTvxa+h0LeyZK+fiRY+JaFHD0fXS5nnNF7D2DVQthlf7uVxEvic6E1z5prSe1X1tKWuliuba78E/TESXz68QgayGXd+K7801X4jYseAeSN8iv9c7by8WQ+SJX8O3k6W1CJX47DRE+iaphPJ4ZDLghxth9AyIbCvVUcYg2WdCXxH3jqyWY2gySNqgGg+QCiF7hVTeNETGVqnwmXdb7bLSDFjxnCSltbr89N7TupTniLCUVqftOWUtfHIJ3LxMKkQVFBQUTsA5NgmBm266ibKysnrLKyoquOmmm8714SicZ7KrB4qh50rI8Qsk1KQ5f0JOZHupYrCWnJ/nV/h74XFLRY76BDdI/pGQvhGiO8tn75vxUp7e6w6pbqlL80ug950y8wjQYYKYU9aIOJZoSR2pEXFAWpF+/hd8NwV+vl9mrXP3yb6jOknVy/p3ZUbbWixVOlqjtC+0uEyq1HzR91645P9kRnbWBNj+pQy4t30uIlBZlu8UGUelDNBNJ6nyMwTIMc++DpY9JYNzlx1+f1UqfRoPkOcyBOAcNYP3D/px+9fbWLgrC4NWc858vRQUzgd6jRp/Q8PnlcahJ7btdbrcHMkv5+XFB5n62SYemL2DbUeLKK06/QjuvDIby/Zmc+tXW7hx5mZmrksho6jK98Zl2SKq+CJxsfh6afRSPTLk3+LHUoNKLW0zQ6dDu6slsa+uiFOXpdOhMEl8dQqSvYWOGtwuOf91niIVLa6GI9ovSFRqqZjx+dqcsPplKMuRuO+6Ik4N5TkihFTmiYgT3kq29XXetldIlWX7qyG8pVRNnsjs1xzubVYMtf42R9eJ2O+0VcfOq6SCdPQ7co7f9oWIaju+kX2c6Hn8I8UnyBc7Z/t+3adKQbK3iFOD2wWLHq9NolRQUFBogHMu5HzxxRdUVdW/AFdVVfHll1+e68NROM/UCDnB50rIMUcSZlKRXn6ehJyo9nLznfrH+Xl+hb8XAbFSkdNpsswMthntbYxssEBocxkUz79TRJQBD8oAe/8vIubcuV5iYq/9ToSXDe9K2lN8TxEzkuqkiYx511vEOZ7c/eIDEdlevCg+HSal9hqdRJjvnC2l6WXZIgqV54rZ4/Ezom2vEi+GH26UmdSiFJkBLjgsA/XYrvUTt+qStELMoBN6+14fGAdqne+KneTVUJGL5/f/4bljHcU3rObVrI40ihHfgphAI3anm9bRiheMwt+XcIueG/r6Nvg16zUn9Yban1XK5W+t4ZM1R9iVXsLivTlc/f56vt+SRoXt1AWNvDIb//5xN7d+tZUNSQVsTyvmmZ/3cc0H60kv8hGJbSutNaf1Re4+MT62FktqVeN+cNsq8Q2bNEsqCxOXiNluSWrD+ylKkRS9yLb1E5fqkrFVUp20hvoJSxc65XnSQtYQaX9I+4/qBLcShclweLn8P7YbHFnT8LZHVkPrUVK5efBXMexviC5TYPdc72Vl2VLFmbZJ/j575sokgAq45Gmp9tkzV8T+vAPymJS10OyShqtyet8p/jsNkbuv4XUn49DShtelb7r4Pi8KCgrnnHMm5JSWllJSUoLH46GsrIzS0tJjP0VFRfz6669ERFxEJacKZ4Ts0moh51zEjxengn84YX4qsso9nBefb0uM3NCmnGAwo6BwqpjDQG8SIccUJoLJ4CdFcPGPhDHv1frPWKJFnAltIYNZrU62z9wu67Z8Do36QEIfEUuaDJLZa0O1D0xEGxko604wE6/WyIB4+AsSqWqJEnPv7jfJ8pCmYgzZZACUZ0n5fc5eGP2uPLaGTpOkPasuTqu0eYGUutccly8MATJ4v/Itec7j37OJX8Nv/9fAYy3gsKI6+AsUH8Vv20dMbVpOywAHfZqG8PK4jrSJCiA6UEkUUfj7otNomNqnMSPaeSeyBZl0fH1LrxN+/gvKbTw2dzdWR/0Jkxd+3U9+uQ/fqgZIyi1n6b76HlqZJVa+WJ+C3enyXqFSn7hVxmCprTC0lUr1XuERMX//dpL4qbhskp6kO4GfjUol5yyH9cQ+KVpDrYktF1lqlVolLUoNUZOq5HY1vI1aW/v+OKpOft522cRI+uoPpWpz8OP1haIuU6SCp+hI/X1ojqsic9mllStnn/wfpFLHUH0tcTslsXDES/UrW9uMEt+1mnQzn8fcwOuxV0rk+YrnpX1q21dQnOa9jSmk4f1qTyKQKSgoKHAOPXKCgoJQqVSoVCpatmxZb71KpeK//23AsFPhb0t2SRUBRi167Vm+YLnschMa35twnQqbC/KqPESYzvHASqUSIz9FyFE4E5RmSbXN5o9rl+2ZKzOf1/8opsAuh0SZDnoUFv5LZixr2DBDjBUzd0or05InpSWpJE1K0VsMh46TYM3/pDpn52zod5/cwPgavLceLXHj9jIpOZ/wlRiH1k142fwJXPYsZO4Qs9C9P0qK1M3LpaLHZZcBteu4FgxHpQhOxkCpuBn9Duw6zrC0hm7TYM3r0Li/HENFrghGwU0hKA6cdvkOqrVipqw1QvZOKWXvNOlYO4UqcQmGwmRiN79PSI+7eX/c/dh0FsL9DajVF9lNmYLCaRIRYOSlcR156DIbyfkVBPnpiA8xERVgPOHnv7jKwb6sUp/r3B7Yk1FKo9CTm/663R5mb264Kmbetgxu7t+EqBpRyVYOJRnQbGhtFUhdNHrxByvNqF3WbqxUf9SgUkkb6qJHxThdo68VAOrSdKhUCWbtgJH/E98dX3SYIKlHTQaKx9fFhN4ilZ4rnvW9vv14SZ+qEdh9EdsdItvAimfkbzL6Hdg9x/e2HSfKNSl9s7Q7XfW++Khd+x3kHgC3Q6pn9s7zndKY0EeuK8eTvVuMjWuoKhLRSG8WQWjPXBGZJs2Sqk+XXT5DfsHy2uJ7+26B0ptlYuR4HFZJPft+qoh4INc5cxjcuEhag0E8m3y9DpB2PPPJPagUFBT+2ZwzuXflypUsX74cj8fDDz/8wIoVK479rF27ltTUVP7973+fq8NRuEDILrUS6n+SBJ0zQUmaXFD9Iwj3k499xvnyyYnqIEkKik+Owl+lMMlbxKkhYyscXAwR7eXGptUVoPWrX50C4g/TdKAIJe3H1cb2XvsdLH5UqnLiesjMsloraSHDX6w/6x3cGAY/Bhvfl0Fxx0mw5TNvEefYcz4tgkkNB36BT4ZIVK1/pFQE+WLLZzB+plTnZO8W49LjSegjg/1ds2HhvSIq6S0yIK/MF2+do+tkP5O+Ec+fkCYyIzvuU2g6pPYmUK2RGxXAb/MMAqoyiDzJTayCwt+JIJOeFpEWhreLolfTUGKC/E7++T9JsevpVMO6T7CtrKtzLOU5cvPc+y5pO62LSi3pQ3XPl2EtJHloXx0fnC7TRMTO3gWbPoIr3/CuFgTZd5+7xdOl8QCp3BnhI10prIWcU+N7yXlHpam/zYWMzk8E8OEv1F8X3koqQZ12Oce2uKz+Nu3Hiz9O3gHZh61UfNq631x/24Q+InCkb5bfy3Nh7ZtSEfTNNeJ5s/9ncNikdff46hn/CLjiVd/+O1D/erVhBox8vXY/B3+VNqw9P+BpOYJMTTw56gipmhnzrnjy1EWjk7h1Xx5v5Tkw9+ZaEaeGinyZTKms9tWxRImZ9PGEtYQBD1RXcf01KmxOjhZUsPZQHptTCskoqsTpOk9jXwUFhTPOOavIGTRoEABHjhwhISGh4cQBhX8UWSXWc2MaWhON7B9JWPXAL73MQ5fIEzzmbBHVUS7wR9f/tcQDhX821jK5kWiIbZ+D3q/2xkWjkxlOp1W8DWoIbiKVKD/cJAPtGibPlrjwH2+Xm5SEPpC6XuJjNXpJLDmyRsSR2O7Vn2vElDi8JbS8DGZf6/vYPB7I2iXpITVeBSq1zI6uekEEIZ1JxKW6bPpQPH/uWCv+AmGtxGNn/y9y7G1GiWDz+ZW1z5O+DXbOkhsIkJuTab+IH0Pd2PYtn0FcT4kbrhmANx0E2786tol6+5cQ17Xh91xBQYFAk46Wkf4k5tT3+FCroH1s4CntR61WMaFHAgt2ZvlcP6ZzLCHmOuOHpJXifbPgPhEOKnKldTS4ibSK6v0hP1HE4vjeco7JOwBjPxDvr8B4OYd8Obp6fytEBL/+J/l/RZ5U+QXGi3HyyP/JPlVaMU2+/XcRG8qypAIntHl1i5YOItriLs9F7R/u66VcmFiLYfFj0GEi3L5G4tnLc+W9DGsh1UiLHpXqzEv+T4SdI2vkWtP+avGfWTpdzqfXzhEvot1zpMW34zVSbWMrE0+0knT46S7v509ZA0OfEmEkvBXEdIWs7bL8tlWwb4FM0kV3lL/J91NhwEPyntf1covvJZ+FuqRukL//5G+l7So/EeJ64EnoQ6YqnH//fJiU/Ao+v7EHWnUsVWMWElSwjcCsDVQGNqWi8XC0wXFEan2MX3P3SfuWL46ug6oCMAVLW1aH8RI5vmcelGdDq5HipVjX6+5PUlhh58v1Kbyz8jAutwiiFoOWd6/tSu+mIRh0F5mwqKCgUI9zHj++f/9+0tLS6N+/PwAzZszg448/pm3btsyYMYPgYCXO9Z9EVrGV6MC/PutwUopSwS8IdEbMgFnH+TM8tkRLTPORNYqQo/Dn8bjAUdHwenu594yeyyGzgVe97y3k9LtfZg9tPlohhj0js4N+QeJZ07if3MjsnSeD+vheYAwQc9Dml0pJ+pB/y4BbZzyxd4K9wvv4Wo2EHV/LTPjmT+W5Fz3iXZ2jNUhrIiroeC1oq9sewg7WptUUJnk/T0Wu3Fgce9/cMktcV8SpIX2T/DQZJJVJGdtr/TRASvLdbokbVlBQ8InH4+Hhy1pxz6zt2I+b/b9rcHMCTac+9GwV6c/AFmH8fijfa3mExcDN/Zug19a5Ga05h5VmwJxpUoEY1kIEhrk3Q5+7RNjRm+H3V2qrBduPlypEjV7Ok/Y651WtH/z2H/nO+wVLip9/lHix7PhaxORBj4qYs3eu/B4QIxM1q1+CgiSJOR8/E6y+280uWFQa8XoJbyP+Li0uE6Po1a9ItWbdVKVlT0nba9/7JPHLUQl4ROiylYr3zRejpSI5/5C0MAUlyLl5y0wIbep9rq1Ba5QY+PSt8O1EqZqM6QLLn60WRELlelFwWLb/5UG45vNaIccQIH+fGq+eute5pOVSkWkMkmNJWYu74Ajurg/y5uURVBRmEliyhxx3IP9bX8yh/Fgah02jMMvOjmUp9GhcwrvXdiFSXSYin71C/vaWaPFh87gBFWz9HIqPyrXWFCJimFoL5khpB3TaoN1V4Bcq4o7+zPivbTpSyJvLD3ktK7M5ufmLzSx9YCBNw0/QEqegoHBRcM6FnEceeYSXX34ZgN27d/Pggw/y0EMPsXLlSh588EFmzpx5rg9J4TySXVJ1yrNzf4mSoyKeVBPupzp/EeQqlQxmjqw6P8+v8PdAbxFvmQO/+F7fbKikd9TFXi5l8H7BtbGp5rD66SzmMAiMlXjZzO0yMN47V8SbK16VG5/9C8QosulgiO8DK16AgQ+Jv8xv/ydGy1EdpAXKF7FdxVi0hjajJMYcZPbbHC5VQXvmSUVddCfZZtULMgi+9L/wzTjoc4/4D2Tt9P08cd2lRaKGxv0h8QRpITtnyyxt8qr63hCdJioijoLCSTiSX8HMdSl8Mq07P+3IYHdGCZEBRsZ1jSM5r5xyq4vgE6eXHyPcYuR/13Ri45FCZq49QpXDxZUdo7mqSyxxx++k6WBvgbYwWX6iOsiNemhzqYioS997RYD4dqIICd1vknPn/oXikeOyQ7Mh8PurtY/J3V/7/163S5x1m9Fyw16DSl1b2Ze5HewVOP2jUblc6DQXSSWE1gC974bgeKnMLM+VVjRrse9obGuJCCtfXSXCRUJvGP8Z/PqIXG9spd7vf902qEnf1N9fcBPI2Ay/PAQthkkV6LzbpO135fP1W5dARJGKAmg7GoIaie/O/DsBcN+0FMfuHzGkrZGqrHZXy/GsfvnYwzXX/0Rs4UbUC+4mqELEQ3+diZcH/peZoV14a10uNZ2FGUVVBFelwdyp8plodYV8DpY8UXt91fvDyNcgIBrm31UrHsZ2g0v/I8dWs0yjg/4PQc/bwBx68r/PCSgot/HW8kSf65xuDz/tyOCBYa3+0nMoKCicf865kHPkyBHatm0LwNy5cxk1ahQvvPAC27Zt44orrjjXh6NwHrE6XBRWOgg5F9HjhSm1BnNAmJ/6/HnkgJQCr10uA46/eMFW+Iei0UBCXyk5zzvovc5ggc7XSdzq8VhLpL2oqkhK2oMay2yxWiszihtmyOz0vNvq3LCoqs2HHbDgXhkgN79Ebla2fg7l+dIO9f31tS1Mq1+G4c/LzPjxlTnNL5EZ1JrWqagOENFWnr/ztdB2jNxUuV0iGmXtkFnTNa9JuT6IyNLlemmJGv6C7+dpdoncLNRt0dKZwHYCfyprsZTrL3/Ge3lke4ju0vDjFBTOIE6Xm/xyOx6PB4tRh7/xnA/X/jQZxVY2JBew9WgRI9pHMbJDNEWVDp77ZR/55Xau7Hh6bSMRAUZGd4phYIswXG4PQSY9Gl8+PUEJ0HK4VAjWZfs3uCd8hfrAz3Jjv/t7ESRqKke+rePXtet7mPiVnANMYfDLAyIcBMR6mySD+LKEt5aqlZoqnvbjpF3GaZNzqq0MNrwLtlK0xiAqne6LR8jxuKDDOPh4SG2r0O45Iroc/s27cgnkHK7zExEHxAz691fET+hknoDHn7tVKrmmrHtLfj+0DDQGeX+dNt8iTg1VRdLqG9lBWoH1/ngGPoy9soTKhMHoO1+D6sAv0splK6t9XKO+YAxA/ek4maSowVFJ0PJHuGXij0zo0JwgZz4qnQmMFvSL7pfrpEYvIuC3E71fi71chMLZ13k/16BH5fpcN2Lc5YDVL+EObkxBs7G43BDur0ejqZ088Dov+GnxN/i2JrC73KQX+qhwquZAdhlOtxutMjGhoHBRc85HBnq9nspKGVT/9ttvTJ06FYCQkBBKSy+yslOFv0ROdfR46NkWcpxV1X3dA44tCjOpSCw8j0JOVEf5N2WNlNQqKPwZQprA5O9EzNg5S8SPliPkJmXpv8UP53gi20uJ/7BnZUD8ydDaQbo5XHwfdGbvWeeSNIkfr1lWfFSes4bB/xavnBoRp2ab7V/JLOr6GZC2QW6Mut8oM5fL/iOD/haXQZepcHgZjP0IMrbIoLdmIB0QK8eEyru65/Bv0PNWWPsGbPtSEkc2fiDpIqZQ6Hm7tEf9eJv368/aJSXuu3/w+ZZ6mg5BFdVJbhgSF4vw0/0meU8Dok/pz6Kg8FfILqni201pfLXxKBU2J4NahvPw8FY0CTWjO9sJj2eApmGSSGV3uVmwM9NrnVmvwWT4c0JGkOkkYwVzGIx6W1o/N8yAygJc8X3J7/045eoEwlqMxVKVjrr7TVJ5d3St+OrUxV4ubTuj3oF51WbqvzwkaUv75osvi8ctInifO+H3/0mrjyUKhk6XVpnvp9UmXflHSkWGRo+qPAs/y1/3PjlXeFCh2jXH2+/FUSki98SvYcvnkPSbnCM7jJeW1B/v8N5J2iYY8pSMwxrCEi0tScFNxF8otqsYVu+Z530dOvgL3L5WzKeDEmq9D48nsp28/4mLIbI9nqaDUP1wE0ZrCUYAvRn3iFdQDX4S1YpnZJKg87XSHrxzlreIUwf/Da9gSegDa1+XBUEJYqBdWQihzcQ0+3hBKry1+O/UFXEi2koKV10Rpw7qVS+yzdqGJ5flMbVPIyb2SCAq0EhWSRWz/kjl641HqbS7jp0XmoaZ0Wq8zwt+Og2toi1sSSny+Rw9m4QoIo6Cwt+Acy7k9O/fnwcffJB+/fqxadMmvvvuOwASExOJi4s714ejcB7JLD5HQk5xGuCRAVU14SYVq1LdeDye82O8bQ6XG9QjvytCjsLp43KI2bG1SGYu+9wrQkPmNhFQCg77bmlqMwaqSuCqD8BtF4PjulTkweInpJKmLls+g8tfhVnjxbPGFApdpki7k1onJeGl3jdsgMyMp2+GrjfA5S9VD5CrTY3bXQXtxojPTspaiOkGhYe926BAZsHn3QbTfpbjq0vNDcahpfI8na+TVgdLjLRGaHLrx5gXH5XqvLAW4tVQF40Ohj4tAtnod6CqWKqOzOHiiaGgcJbJKbVy65db2J1RO7G1dF8OqxPzWHBPf1pFWc7j0Z0aMUF+tIm2sD+rrN66WwY0JcJyFpMqLVHQ605odzV2h4PvdxWT4IkkvKIIfaA/noD2uFPWot46UxKDarxV6pK5TSr5ataVpEuiUesr4bLnJCjLViE/jfuJSXpVkZx3Vhxn2FueA3Nvkdhpp82ruuJCR+W0Qf7B+iuKUyFlnZxrh06Xc+Ovj0qrrM+UMbe0trcdC/t+rL964MMi2kz+ToSi/QvE66b8OINij0cmIda9KefpGqGtLi1HiK+RfxT0vhOXw4rm48HeFTz2CtQL7oYbfoHbVksLWU0FaEFS/X3WvB+FyRJXX/d9mHebpFqlrPF9zQ2Irb/PgBjfn7tj+z1KpL+Wggo7b/x2iFWJebw9qQt3fL2VvZn1zwsL7+1Py0jv80KQSc+jw1sx4cP6sen+Bi3D2vpI2/KFxyOfYbdL/IqUCnIFhQuKc35Feffdd9Fqtfzwww+8//77xMZKROSiRYsYMWLEuT4chfNIdqnM0IT4n20hp3rWpk58ZLifCqsLCqynHoN6xonq4J2soKBwKpSkSyvVrm/hi1HwTleYdY2U90e0hcydsv6q92Q2EGSmetDjMOQJ2P6FDM7XvuF7/xV5YrpYl8Jkadca/zl0v0X2nbkd5t0KC+8T4+7IdmIiejyVhbD+bRm4vt8X3u8t1TsL7xXjx73zxO+g8LDMbvvCViYD5fA6Pf0qlQzAa6gqkhaGlDWSntVmlAyYx8yojV23RMNlL8hxjngZd49bpSoIcDceiOvmFajCWsi2erP4BAVEKyKOwjnjYHaZl4hTg83p5tUlByizOnw86sIi3GLg46ndGdC89jxi0Kq5c1BTru/dyNug+GygUoElCpclmk4xJvpX/kabxZPw+6A7mtmTUOORqrySdK+Way9qBN8a3E6pyFlwj5jGx3aRFL/VL8OHA2Scsf4t3/tyWsVY13gO/ADPJBpd/fen2SVyTk3fBF+OgpnDYcc30G2aCCg+UYPOIJ5DAx6WiQCQKs+xH0hq1Mb3oChZfHTWvl5fxAER1dVqEZAqcuDqj6XCFMT3bdDjUj1Zkgo6I87ktai2fNpgG5Zn7ZuScLX9a/hyjHi71VwzfRHWon57nbVYJjGcdghtUf8xJWnyOK9lJ/jcAYQ0Jb209nueVWxlf1apl4hTg83p5rWliZTb6p8X2kQH8PbkLgTXSYZtFu7P7Nt6Exd0CobK5bmw5VP45BJ4sx18fbVMPl5spt0KCn9jzvnoNCEhgZ9//rne8jfeaOCmQuFvS2axFYtRi+FsD+qKUmTgUOemL9wkGmZ6mYewMxMQcPpEd5LS39LMMxI1qfAPoDQTMraKwfGu72qXux2Qu1tmRTtOgEZ9xGTxkqdFgFGpYdWLEt/d+VppZ6opDz8el0MG05HtvVulDiyUWbl2V4mZZU35eVURbJwhFTGj3pZ0mONpf7V4HNTgsMqge/NncKjaz8IUWn+QXJeCJJnxrvEDajFcZoXrotFB5ynwYX8Y+Jh44XSaDCNelvJ5nRE2fQxLn8Q25D8cbPMQLXrdh1rlQWMMQGtWUhMVzi+/7PYdtQ2w6mAe5VYnFqNvX4wLibhgE+9e25WCCjtVDhcBRh3hFgPGcxh57Oe20j71a1R1z3W5+0WMueT/JCa6563eKX417JwNg5+QSOvjGfJvWP+uVI7UoDV4m/ceT14idJn251/M+aAiT1pM//hAxCj/SOh5C3w3pfb8X1koAnxcD6lW+uVB73007i8TCT/cDIMek/d62H8lQaroqCRg1bxv+Qeh8UCp0vT1N2k7BvxC4Otx0pYU3Bi63QADHxGxJroDvNsDbl0Jv7+KetATqPfMbvDlqQqTRICrMbJOyoPed8LmT2pb4+rS/WaJYz+eoiNQcAj6/UtErbqtWfmJMpFgDKz1Cco7AGHNvZfVoaDnI8z4o7aarVN8IIv2ZDf4OlYdzKXM6qznl2Mx6hjZIZrujYIprrSj1agJMekJO5WKuKpiaaHb/lXtsqwdMnk04SuZKDkf1ewKCgpenJcaz6SkJKZPn87kyZPJzRXVfdGiRezdu/d8HI7CeSKrpIow/7NYYl1D8VExJaxDuEkuQGml59EnJ7KD/Htkzfk7BoWLi4ytYjRcV8QBGWCuelkGt1s+gzk3wDfjYfa1MvCylcvAuKqo1iz4RDOCeQfhildkQF3D5k/FXHjl8749BNI3yaxoi+G1y1Qq8ZKoiRavQW+CiNa1Ig7I7F9QQsPHFNZCZjcBWl4us7u76gzS/SPFjydxiQhO698SL541r8HWmRJVPmca7PqWyu53kpZwFdN/TaJYG44htJEi4ihcEAT5NSzSmAya89MK/CcJNOlpGu5Pu5hA4kNMZ1XEcbrcpBdW8uP2dF5bepDl+3NwluWiaqhKZu0b0iqVvln8wgx1WlMCYkR0CG8j63R10rFCm4kXTF0RB6orL05wTo3pLP4vJzP9vZDQ6ETAufpjaRHqOhXWvtnA+X+zjLNMIbXLmg6W1qvCZBHNPG6p5PzpHhHIlj3lLX4FxMGh3+DyV6DJwNrlKhW0HgVDpst5vcZbpihFouHnTIMfboRDy6ViqKoYTCGoStMlDr4hItp4C0YeN6x7G8Z+KNWbNRgCYMRLknDlq1IovLWIUps+hKve9x5vGoMosaspmfhjbWUowOpXcU/6VnyBatAacQx5mkWVbbzaEq0ON5YTmJ2b9BpU+D4vaNQqYoL8aBsTSMtIy6mJOCCvs66IU5dFj8pn+U+QXWLl98Q8Xlt6kO82p5JaUIHd5Tr5AxUUFHxyzityVq9ezeWXX06/fv34/fffef7554mIiGDnzp18+umn/PCDbwNKhb8fmUVVhJzMvPBMUJgi1S91MOtUmHWQXn4ehRy/IJlRSvldYo0VFE6E2w2HV0B8z/rrzGG1SSG+KM2ASd9KiorTJmJM3/tk8Hs8OpMkvxSnyuBVpZbZSZ1JTCZTNzT8PIlLZEazxy1gDJD2pB3fiJdBXa+agiRvA00Qw+L+D4qJsccFR9fX3vQYgyQSOLab9Ol7XKA1w22/Q0W+3Gzo/WX283B1tLi9Qm4ADv4qz5W+GefUX6h0qZh/2EW8zUznuMCz39qpoHAaXNUllg9/913ZMaVXI0KVz2s9XG4Pu9JLuO6TP6hyyE2hXqNmyxQjAcebz9Zgq24P+eNDSdEbM6PWD6vgkCy/7Fm50b/6Yzn3aQ1yLkzxMfmy/SupyFj/trS4WktENHc75dwU3gYVHmkTvVharPyjYP07kLxSqpNiOnvHsB9PynqY9ivYyyTF6eCvMgFQw565IgZtfK/+Y4ObSGXpoUUi9F/+qvjWqDUyQaDRi3CTuAQC48Vg2lklE2E1pv7JKyROfvvXENcd1VdXiQn+zm/ri08qlXgpfXed9/Ijq6Xd6pKnIKSZeMSoVHgqi1Atfrz+cZtCKYnuh+fqrpSXlRIaGYf12sVUFOeBx02u28LrG0pJLS7lyYEzGRynori0nEICmLfTSY9eX9AmwIbabceqDybDGcD0L3d5PcXG5AKmj2xDfrmNrGIr29OKvdZP6d2IsDN9Xsg9wcR6jSB5mpXkaYWVTPn0D44W1KZI6jVqZt7Yg56NQy4KI3cFhQuNc/6tefzxx3nuuedYtmwZen3tiWfo0KFs3FjflEvh70t6cdXZH5TaSqGqECyR9VZFmFTntyIHpCpHqchROBXUarkhqDtzXINKfeIyZ49bvGzWvCaPb3WFeMNc8T/ZZw1BCXD1R/Dzv8QsuCJXhJg/PpQZV7XmmKeMT4wB0o7lqTaNPLJGxMq6M9pxPaDZEO+ZW60Bul4vVTUGi5TPj3pLWsPCWsgxzb1FnnvxEzLQ3j0bljwJBxbId3zhveKx4KiT1OW01QpGFXm4Dy9nYaqe0KAg1hzK447Bzc9+a6eCwmkQE2TkwWH1KzvaRFu4vk8jdBeRWe65IqfUyk1fbD4m4gA43W6cGuOJH6iprn46vBxWviAi8fw7YeULeIY/L/43Gr1U8x1ZLQJBWbYYKh+P2wVx3UUgV2ulSmPSLBELxn0qQrXWIOfqiwVPdUhE3gFpR8s7eOLzv7m6PfaPDyWmPLariO/GIFl/8BcRubpOlfeohpgukoIV3VmqLTO3wd4f5e+TsVW8a/b8INeXqz6QViqtXip4xn8q8eYAhkBp9Wo6GNa8LqLbpo/E061ulYxfsBhdW0t9V0jl7JGqoYo8EbFS1oHBjGvKfG8fuPBWZI2dy0FrCJd+lUP/r0sY/20aOapw7l/tov9XRVz9TSprk4tJLaxkfY6Wfe4Ekg2tGDsrnU82ZHL7/HQGfpnH5XMqqDTF8ulG79CArvEWNt7RjCZJX/GufgYzW29m+Y0J9EiQcUDbaAvX9Uqol1r1lzEEnHi9+vTaO8utTp75ea+XiAOSbHfLF1vIKfORsKmgoHBSznlFzu7du5k1a1a95REREeTn55/rw1E4j2QVW+nWKOTkG/4ViqqrFPzrCzlhfmoyzmdFDojh8YGFkqwVFH9+j0XhwqfDuGojz2jxufEPh4JkSN8CTYdA0or6j9Hoxe/GYYU+90hZdEgTucHQ+cHk2XKDUTOrueNbad+KaAOz6lSK7flBZmU7T4HNH/s+vrgeMHuy3NRc+h+I7y03QDctEX8ftUa8cL4YDaPflsG82ymizfZvvGe6d3wDHSaK786Pt4sYpPUTw+ZZE2S2tgadScSe4CbS1w9Sxl7m7SugP7yYoVdMJKXEw60DmhJ9KoaPCgrnkEA/PdP6NuLStpHM25pOUaWdkR1jaBsTQFTASYSJfyjpRZUUV3qbvbo9kFTlT4g5TKr2jiespVQUTPhSzpGWKLDbYNgzVAU2x6DTotKZxG9s1oTaNKYjq2HqAhEZaqoMDRbxDfl6vLfP16YP4cq3RIxuNgS3PgCPKYyLRjrWaCGhTgXogV/EM2f71763j+kiLb015sIb3oXhz0HHiXLN8HhEcO88BSZ+Je+fOVzEF1upPNZeUbu/dW9Km1PeQanmMYXA+Jmw8rnaFqetM0XI6X2XxIfPvUne8/IcWX9oqWx76X/l76Qz4dHoUK18Hvo/UHsNOp7ml8r1KGsXRHdC9cUo3C2vpOLG1ZQW5OLRGjEFRZJW5cdjc3eTXy6eOnszS0nMKePuIc0pqnSw9lA+Bq2awa0iKKywcSS/grlb0vjp7n6sOZTP3swSujcOYVDLcGKD/HhlXCf2ZJawaHcWLSIt3NykEN0XgyTNCwhiLkG6l/jkmrkc1PaiUZg/kWfjvBDWQq6rjsr66xL61BpWnyKFFTaW7/fRlgZUOVwczC4jLtjkc72CgkLDnPOpgaCgILKy6vdWbt++/ViClcLfnzKrgzKb8+xHjxelyM1jncSqGsJNKlJLz2NqFdQmLtT4ligonIigRmCvklL/ilxIXCoGvk0GiQHn8aXOKjWMfF3ala6aId4CnSbJwHr1y1L27rKL4LH6ZVj6lHxX+j8gs6DHs+s7SQUJ9+E7MOTf4htR08rw23/kZscSLQPl76dC7gFYOl0G2VtmwuUvQ1RHKM/z3a6w+zsRY4c9KxU7+QdF1Kkr4oAMNhc9JjOoHrfMJo55T0wr6+A2R6BSa2keaVFEHIULlkA/PW2jA5h+ZVtem9CZoa0jFBHnBJRafdyIA/9ZXUjlVZ9LpU1djEFizPv9VPn5dhJ8PhIqcyk7spkvdlVgdaug6zSpAKwbqd39JhEyLn+ltrqm771iJn+8WbvHI9WNdokpVzmrcHouGhkHm8qEpyBZKiNBJp3ajvHtBTTsvyK2HJ8QtWS6tJH3vV9+93ikDe3byVI1U1Uk14kf7/AWcUAmFxY9An3vkd8rC0Wo73Fc7PiGGdDqckluHPcZGIO9K5+ydkil1XdT4NtJeEzhUtW55TPvv2MNgXHQ6w6Z1DCFHvPk0SX+TFX6bib/amPA59lc8/VhEnPKuaxtJB9d3433ruvKO5O6EBloxOHyEOSnY0L3OAL9dDz4/Q6e/HEPNoeLO4c0Z1dGCeO6xfLmpC5M6d2I+BATarWKqEAjl7aJ5LUJnbmlsx+6uTfUF1McVQQuvImOQdazIuIUVtjIdAXiHP+Fd+UUSGXT6HfAdHqecnaXG/cJhtuFFT7MpRUUFE7KOa/ImTRpEo899hhz5sxBpVLhdrtZt24dDz/8MFOn+kgHUPhbklUis/9n3ey4OEWqcdT1B0/hfip+K3Pj9nhQny8DSWOAtJ4cXQudJ5+fY1C4eNAapHe9bjJI9i4pY7/2Bxj7kZTBZ20HUzg07gdbP5eZVIBr50h61NLp8vul/5F40Zr1AC6bfC49PkZdLYaJv07/B0WcSd0opebNhsCBX+vP1B5eDgd+hm06GPmatHEdWS3rEhfL6xn9Niy4t+HXvO8naVlY8SxM+kYqe3xRmgGoYOhTIg65bFKWX4fyzjfjUuuIOlXDRwUFhQueJqFmn8v3ZpbzzqEEHrptLerDS1Dl7kMV1kKE6OX/FR+wGuwVuHUmFprGUVJsQ5W+GVTO+tUa0Z1g9nWS2nPt9yJAtxgOq17yfXAet7TmNO6HuzgNApv43u4CJMeqJiGyHWzZCNd+J+f7xCUSGV6cJudwvyDocI20ju34xveO9v0k145GfeT9spaJ55lfEOjMoFY17PFWnuvdTpyxVSYajufAr3LtS98E036Ras5d39bfTqPnQBEEj/mesLL9aEvT4MbFqJJXQEkGNOorlSgL7pUqofZXwy8PHXt45N5PGdvmv7y9rork/AraxwayaE8WH/6eTICflq9u6sUHq5NYui8Hl9tDqFnPrQOacsuAJry9/DDf/JHKpW0i+X5LGma9hoEtwxtMofOU50lUuS/KsvGU50JInO/1fwKny83B7DIem7eLPRmlXNYqiKevW01UxjI0RUmomgyUapw/UT3ub9ASGWAgp9Tmc3372IvEN0pB4QLjnAs5L7zwAnfffTfx8fG4XC7atm2Ly+Xi2muvZfr06ef6cBTOExlFMqN+xg3ajqfwiM+2KpCKHLsb8is9RJjPYxJIRDulIkfh1KjI9R1/6nLALw9IeXloU2lxsldItc6Ah6WUXaMFU5gIKyCGm8GNpXKmLlpjbSoISAVP26ukcicwDn59RKpiAuNgyo/w60MiBrm8WxsAGQjrjJC2CazF8vx1BaJ9P0k5/vEzscfvo6ZM3n2SVkiXTQw2I9rCile8VpX2eZRSUyOlfFtB4W9GmL+esV1i+HG7t8ir16i5vGM87d7eQIe4rnw8+nKCf7oecnwbuVZUVHB5q2ZYPOVo570GPW/3sVX1WGH/QhHA43tJu9HxlSh1sZeCRo/HWorb7YaLpLkq0KgCl1G802ZfJ+byTitUFIi40mE87PpeBJCtnze8o6pC8aXZMEMMis3hkLhIqmIaD4SYTg0/FuTaolLXvse+DKxtpdJqbK+Qa+QVr0LGJvFqq0FrpHzcN/yS5MJgcBLm34EOcf1IsHjwD8tEk3dIzJxrkrR63iZx9VVFdZ6nHHMd3SWn1Mq6wwUAPDa8NY/P2+WVOFVQYeelxQd48oo2dGsUTEG5DaNOTVaJlbtnbeebW3rRr3lYw6/7BHicJ15/uqQXVTH+gw3HvKaWHixmWWIxPRv35o2JdxDzF6pYIwOMPDWyLfd8u73eusGtwokMUCZXFBT+DOdcyNHr9Xz88cc89dRT7Nmzh/Lycrp06UKLFi3O9aEonEfSi6vQqFUEn83UKo9bZnmaDvG5OsIk5bRpZW4izOfRgDCqg5gAlmaedgqAwj+M/MMND+4Kk8XXJnMnrH1NUlh+fwWSV9Vu03gAjP8M5t4sM40FydKGVJPgAuJHMKB6BnL4C/Id+v56ed6EPuIdkLFVZgr3zRfRp6Fjiu8J276Q/2dsk0F7SFPvyNn0zfIdrbusLi2Hi/EliOGz1ljr51MXnZ+kdw16VFq1Bj2GO2sXDpUeV7NLKfQEEh4aUf9xCgoKFzWBJj1PXtGW5hEWPl6TTHGlg/gQP14d35HZm1KxOd1sSSliX7Y//SryfO9EpcIcFo/6p2nQdrS0dMZ2Pe6J4qW9NTBOzn8et6T4OW1y7s3d73vfjfpJklNMF/wMp2cSe16pKkHlqJTz6a2rxCdtwzu15/ugRtIeW5Ih71XGNt/7ie8lrU1jP4SCw3LO15ukJW39O+LZpjf7FvQ1+uqkwmoRxxLtPdFQQ9PB4nOUtFKSxkozxBTZ4xHzYks0zkYDeW69k0vbhWPQqXnx1wPsyypFrYLpQxMY0/UeAtJXocWFKrojJC2vl9JV0uxKViVLG1CAUYvDJcflb9ASaNJ5iTh1+XhNMo8Ob8XO9BK2pxYfW/7Cr/v56uaehJh9CBnmsIZ9arRGNJYzdz1zutzM3pzqZRgO8vb9caSQt5cf4j+j22HU/TkRUqVSMbBlOJ9M684Lv+wnOb8Ci0HLtL6NmNqnse/Xr6CgcFLO291rQkICV1xxBRMmTFBEnH8gGUVVhJr1qNVnsRKmLEsMXv2jfa4ON8lzp5Wdb5+cdvLviWKdFRQAOMlnVa2DpU+Kt8Omj71FHJCy9nVvwbSfJVrVWSWtTVe8Wpss5agUUaXnHVKF88eHtQP31A0yYK8RHLd+Lm1WPloXaTxAbnZs1QPbDtfA1i9g2DPeCVuJi6HNlTJjezxBCZJkklk9i7dztgz+fdH/QTFMnjURFt5HvtPIhpipTN3fncPuaBrHxmA2nPO5CwUFhZNgd7optzlwn8hE4ySEWwzcMbApi+4fwO+PDmHuHX1pEWFhc0ptNcVrG0spGPi8z8d7ut+CetcsOSeCnPv0/tJCVcMlT4l32KDjYqiProMRL/lOpGoxHPyj8JTlUqE7y+EOZxiP2yUiSqN+Eu299nVv0b74qFRnhjQWj7Tj/VSg9jpQVSTv6eqX5f068ItU81DtmdP/wfqPBaky3T2n9vfBj0sFaF2iO0s15q8Py3qQqp0f74BlT0lF6KHf2FOkZfbWbCIDjTw1fw/7smQCw+2BZ5Zn03dmDh+5RmHtcC38/IBc++piiSI7fiQbkuUz9cCwlszelAZAbJAfh3N9CEzV5JXZCPDTcVnbSJYfqDX93ZtZSpVdPv92p3dVly4wGseQp3zur6r/YzhNDVTy/Akq7C7+SC5scP3mlELKbb69qE6VAD8dl7aJ5Lvb+/D7o0NY8sBA7r+0JRGK/5eCwp/mnI9qXS4Xn3/+OcuXLyc3N7e6zLSWFSt8pK4o/O1IL6o8+/44RSnyb4CPqFDAqFURoIf0svOcXOUXDAGxcHSDlGgrKDREWAvvtJS6hDQV3wKAhF4y6PbF0XXSqrS0TitrXA8Y9zF8d73MfK54Dm5dKSkix/PLwzD6Xdj/M+ydCzu/g+vnw9o3Zd9+wdD5OohsK7OwIAaJziqI7gCHlkjCy4YZEjPrHwmlWTBtoaSUHFwsNwQdrpGbqJK02ufev1BifUe9DX98AIVJENocT9/7UBmDYM1ruBoPIrf3kzz7h5sruzrZkVqEQXsRRf4qKPxDKKmyk5Jfycx1R8gutTK4ZQQjO0YTF+yH6k/41mk0aqIDa9s/yq1OogONHKq+wd6WWsL7kY24bdw8IjY+L5UaAbE4+j2ETqeHH28DVBDdUUSZqiJoOUJ8cfbOFx398G9SKXHNF7BxBmTtlGt3dBdJ51v+jPi0mMPFlLftGMg7jCu6C1Uqfy4mJxCjVg0Z+8AvRM63vrCWQHE6JPSFGxdJjHvqhjrXgXYw/w7ofjOsexsu/T85x++dJ2bD4z6Dskx5r8fMkOfJT5QEwv4PyPI/3heftH7/kooc/yipwNT7i3F/o74iKNnKwD8ST0I/PE47ar0Z3C7sMT2p6nort36ShJ9OQ5nVQUpB/SoXm9PN2ysOM6xdP6Im/YRpywy0e38AVDjbX0NWu1u5c14O7WICeGhYSxwuN2sPSyJaUaWdiBN4rxm0ahJCTNwzaxuuOoJlqFlPRnElD81JJCrAyI39mtA4zESgn1485DpOxBHUGN3q56HgEIQ0pWrAk3ji+2A2Wxp8vtNFr1URHWRke5rv9ZEBxjN2HQ1XPOoUFM4Y51zIuf/++/n8888ZOXIk7du3/1MX6+OZMWMGr776KtnZ2XTq1Il33nmHnj17nvRxs2fPZvLkyYwZM4b58+f/5eNQOHXSi6rOjT+OweJtlHccESY1aedbyAHx9Di67nwfhcKFjjkChr8oM4910ehE3KhJmnL6NhQ8hqNKqmJq/GrSN0Pyaok0rzGw1Blh6HQpeXc5RFxxVomp5OxrYeQb0PoKSfUoSZcWwUv/A7n7pJ1qzf9qn6/JIGnjiu9ZPWvtEf8Jjwsq80XcqSrE0+9BVL3vlu/u/p/g66th9Dt44nqI+ShIqXtoM+h1l5g5Jy5Gtf1rMge+ytGRy9ia5eCzOYUYtWoGt3MyuFUEgX4XUTuDgsI/gHKrg+82p/PCr7XtSBuTC3l/dRJz7+xD84i/fpPqb9Ry5+Dm/H6oNn78k835/JJo5OEBbzJsdACrDxcTZfCjx8/D5BzXZjRo/WDAo1B0BH66G2K7Qd/7pAW1+00iHLhd0P0WOY9WFgAeyDskKYEqlQjiRSkw/14Y/RbZpTbc5gtgrHEa6LBLu1h0J3mNnSaJSTFIC9PeeXKtyT8gIkOTwdDyctnOXi4pVmtfg/YTRMhx2cQw2V4hSYQarSRhNeoPCa3FZH/MeyLsmMLkmqQzSkpSwWFY9KhUjo77FNpdJdexfQtEUKu5lpXnUHnZK5T7xVI8fgVBJgPvbCyiT64Bh9tNsEnHkbyGPdlsTjelVU6u/Pgwt/a9kZtuuJ99mWVszPYQlK7n3ktaoALCLHpCzAbeHhVLjEVLSrGDyGAT/gatz8qV0Z1i+GxtMknHPffkngm8viyRjdXVMPN3ZDJ9ZBsm9UzA36BF5x8Kba/AFtsNnHbcGh1+Qb4nJ/8Kfjottw5oyq+7s32uv3tI8wZNmRUUFM4f51zImT17Nt9//z1XXHHFGdnfd999x4MPPsgHH3xAr169ePPNNxk+fDgHDx4kIqLh/tGUlBQefvhhBgwYcEaOQ+H0yCiuommLM1cW6pPCJOmZPgGhfipSSy+AwVVEW5nps5aICa2Cgi/0Jug4QQbWa9+Q1JXY7tDnLhn41rReaU8y46Xzq59KtXM2XPO53LS0vUrapjJ3Qv/7Jab88G/yuPbjYOJXUJEPqKVsvapIbm7WvA5dp8gstqNKRJu2V0lKTECsCDw7Z8uAvvkw6HGztHpl74J9P1Ew6guc4e0Jy92PtjAZEnpTZQjFeeWH+KeuRLXrW3A5JWo2IAq+GAUVebibXcq3eyt4Z52YIodbDLxwdQe+3niU6SPbElVnll5BQeH8k1du58VF9T1lSqoc/GfBPmZc1/WMCLBtoi08OKwFb/x26NgpL6fUSkRkNFd9tZfk/AqeuiSKHgl9xNx2zw+SkDfwkdprccZWOR+OeFHaVX+6GzQG8RlrfaVUoATEwrbP4ed7pWKk+03SVtT7djxuJxqNDo+v1qsLGY1exiYuO1z3g7RALfyXrGs9EibPhp8fFO+gLZ/Kz8jXxdx474+ACm5ZKcbGM4dDVTE07i9tWKtfgcPLZF8tR0DaZtgzBwz+0hockCDn+aPrpHLHXgkdJsj177spDXqqeaI6sTo/gFWHU5nQLY5liUV8vSWLLenlvDu5Cw/P2UmQueFJRINWTUmVA7vLzYw16ThVTdmTWXnM0BjgrsHN8HeX0FS1j9HbX4bCJLqHtqDI/zEW3tKTMZ/updRaK+Z0bxTMTf2bcM0H672ea1jbCJqGm3l35WGv5c//up9L20TiX6cV2BDoO7TjTNIs3MxTV7bh+V/2H4sKV6ngvqEtaBsdcNafX0FB4fRReTy+MmbPHjExMaxatYqWLVuekf316tWLHj168O677wLgdruJj4/n3nvv5fHHH/f5GJfLxcCBA7nppptYs2YNxcXFp1yRU1paSmBgICUlJQQEKCe2P4PV4aL1U4u5bWBThrQ6i+ajc6bJIKTV5Q1uMmufne05LtZed+ZKVP8UJRkw/3aYMlfMZBVOyj/yu+jxQFm2CCdqHWh1UqXjqJJUqMJkaYfq/wBk7xbx5XhqIkRXv1x/3T2bxTDghxtkNnni1zJoPt5sMawFTJwFX44WLyoQj4iqYqmycTqgRbUp8u45clO0/L/1B98GC0z4Er6dBAYLORN+oaLKhsWgJtDsRzl+7CnRkZhTzsg4O9GHZ4s4dOR3qSKqpnLqErY6m3Iot4LmEf4E+unQqCHQT098iJJSdbb5R34XFf4S321O47G5uxpcv/qRwTRqIFb8dCmzOigot7MjrRgPHlpEWPB4PIx6V6pgYwKNrL4hGt2348Ukt3F/MS+O6SpeY7l7RbT4fmqt51cNLS6DbtMk1el42oyCNmMgqj3O0kwqo/sSYD67ovKZ/C6WFWRicRZCVYlcE8pzvTcwh8PYj6RSc/a1tcvDW0HHSSJy/XSX17kakMqnG36V97I4BWK6iZhvDpMJidT1cr3ofrNc51pcKkb/B38R0QgP7JhV/4Aj2uC69BneOxTCuxvy+O72Pryy+ADrkwrQqFXMuqUXOo0ag07NvbO2k5xfvzLnul4JlFmdLNgpCWhNw828NqoJAe5i9M5ynDp/LMHh6Hd/S8CaZ+o93jnseY42nczWjEoKy+00CTeTWljJ1pRCHh3RmuT8CkoqHbSJtrBkbw4zVh7G6cMb6rXxHRnX/fRjvv8qFTYn+eU2dmeU4HJ76BQXRKi/XqnGUVC4QDnnFTkPPfQQb731Fu++++5fbquy2+1s3bqVJ5544tgytVrNpZdeyoYNDRvHPvPMM0RERHDzzTezZs2av3QMCqdPRrGYCZ6on/gvYyuVioGTpECFm1RkVXhwuj1oz6bx8skIiAFjkJQdK0KOgi9cdkjfKolTpRmyzC8YLq+OSz26VjwZLvk/8Z8Z/6mIHoeW1u6j6RDofQf8cHP9/Ud3hsIUqCqQ9qiuU2Hbl74TM/IPyUyprs5NSdom+X3HN/L7zm8kpSRrJ5Sm+55BtZXBzm+h5+3Y208gdNN7RO6oTrlSqQhqO5Yug/5LSaWBlRlOhoX3JnzpPVCTPGMIoHjoS7y7XUW2LZ2YICPD2kYQH3JmbgAVFBTODjVpPw1x/BRjXpmV/HI7FTYnYf6G07q5tBh1WIw6Qv31FFc6KKq0e/mtRwYYcCatQldzXs07AH3ukfPnqDfF02Xnt/VFHJBqkiX/9v3E+xfCoMfAYUdrDsNgzQVzo1M65gsBDYAhUPzQjhdxQM7D6ZskjdAcVl2lCZTlSIpU7t76Ig6A2yltwI37SavuiObij1aYLP5rf3woH4BtX0gb1a7vpRJnnx12fQdXfyxBFvvm16ZZxXWHwU+gmXcbo8f8yNvrPWQVVzG5ZwIHs8t4cmQbVh7MZVibSFwuDx9c342Hvt/J7owSANQqGNUphh6NQ3jw+x2AVOe8e2UUrbZNR3Nw4bEPpafV5ai63uAzUUq76jkCEobz+tJUDDo12SVWwi0GPpjSjcahZpqG+wNwJL+Ct5Yf8npsVICROwY3IzbID4NWTUp+xZ8WUQrKbRRU2CmpchBi0hPmryfwFFJizQYtZoP2jImo55riSjv55XaKK+0E+sl3XknEUvg7c06EnKuvvtrr9xUrVrBo0SLatWuHTud9gpo3b94p7zc/Px+Xy0VkpHfJYWRkJAcOHPD5mLVr1/Lpp5+yY8eOU3oOm82GzVbrN1FaWnqCrRVOhbRCufCdVSGnIEn+tZxYyIkwqXF5IKvCQ7zlPAo5KpWYuKb+cf6O4QLnH/9dLE6VChiXvXZZVRHMu0XMg1Vqabcyh4ungDlcElN63Cw+BlqjRI0vfqx+fKtKBf3ulxaB4qOyLLZ7reeOL5KWSxtWYTJEdQQ80GWqRNJq9HDgZ5h3qxzL1s/lMVqjzHZrjZJEVZoByauouO4XjGteRLOvzvnf40G9dx5mj4c2fV5kUQEkR3VHdd0yzM4iNHg4VGHk5bXF/J6Uzwtj27M9rUhMIhXOKv/476LCX6Znk4YTnDrGBRLgVzs8Tcor57YvtxzzF1GpYGznWB6/vPUpJ97klFr574K9/LpHPEA+ntqdqAAj2aVW+icY8Du0sHbjmkmgmZdL1PiEr+GLK33v2BwmZu5RHeXcmbXTe336ZsjeC0FxaFqcGUuBupzN76LBlg9GCxz8teGN0jZDkwFw4xLxyTEGVZvbWyFlbcOPS90Ave+E3/8Hc2+Bq96XVquwlmKUX1Uo1625t0DPW0Hrh2vkm3jcTtSOStR975NJibJsaSXO3iPbVhURUnGIJ0Z0oajSTpvoAKZf2ZZD2WUkhJi4YeZmymxOQsx67hzUlGfHtKfK4aTS7mJ1Yh4Pzdl5rK3o3r4RtNr2LJqDP3sduurgIjm2PnfJ8dfFUUWYuow5d/ahsNyOXqsm1Kyv9zkN9NPSLiaAvZny94oN8uOFse155uf9JOXJ9Vmlgqs6x/DE5W1OK9kprbCSu77ZdkykAhjSKpwXr+7wt24zziqu4pEfdh0zoAbo2iiItyd1IS5YqcxV+HtyToScwEBvz4+xY8eei6etR1lZGddffz0ff/wxYWGn5s/y4osv8t///vcsH9k/i7SiKtQqzq5KXpAkPeym0BNuFlETQV7qJt5ynvvXI1rLzJPLKSaACl78o7+LbpdEa9cVceqy5VPxFdD7SwVOUIKkWB1ZDYlLarfzC4arP5LP2b6fZH8RbWDAQyK8GCy1vhDOKtlfVZHv59RXbzt5NuTsleQrjVbMi6uKoc/dENwYMrbIdn3uFh+dw8tFSBr8uMSWb/wAj8flLeLUQb1/PuH9HqdDbAgTPxahc3y3OML89XywWqp8OsYFYtRpuLlfUwIUY+Ozzj/6u6hwRoiwGJjSuxFfbzzqtVyvUfPcVe2PjQ+yS6xM+eQPskqsx7bxeGDe9gxC/fU8Mrw1+pOk6VQ5nLz1W+IxEQdg+f5snr6yLfd8u41yhwq3zsKxvcR2E/HCXgF5B8XTy+AvaX91aTxAzrONB0graptRMORJSf3L3i3bqNQQFA/r38LT/HLsTvdJj/d0OJvfRZUhQKpu9P71VxoDxTNIpYZtX8m1o8M1Iqxs+QzPwIdQnSBoAp1fbfpiRZ4INx6ksnTgY5JQdeAXSbHaMANP/iE8HSejTlwkAtDHgxvctb/ZzA05n+GqKsIVMpEcWxA9mkRx0xdbjlV6FVbYef5XmfBd8q+BvLhoJ7vSS7z2M6q5Ds2sn4/fvZC0Anrd4ft90+iIDzYRfwLxIMRs4IWxHbjmgw3YXW4evKwlT8zbTeZxn/Mft2cSajbw6IiTf84B8sts3P7V1mPR6jWsPJjHsz/v5+VxHfE3/v3GlyVVDv49f4+XiAOw7Wgx9367nU+ndVcqcxT+lpyTb/PMmTPPyn7DwsLQaDTk5HhfXHNycoiKqm9ym5SUREpKCqNGjTq2rCb+XKvVcvDgQZo1a+b1mCeeeIIHH3zw2O+lpaXEx5/7vtW/E+mFlYRbjGjOZitTYRIERoP6xBe+MD8VKi6ACHKA8DZSppuzB2I6n++jueD4R38XnVaJ6m6I/ERpJ2w/DlLWiJCzZ55U49QVcqqKxPOm3VjxKLAW165rdbmIn1qjmEvuWyDl7Gte8/2c7a8GVNLqNfI1mX39/ZXa9bu/F5Fo4jdS3bN7jrePxK7vIbI9njHvYa0ox8etguDxoHOUs/2omond4xjYMoIAPy3Tf9xDiwh/xnSWcvhwi+GiLQe/2PhHfxcVzghBJj0PXNqCAS3CeH9VEvnlNno1CeHOwc1ICK29AT5aUOEl4tTlq41HmdanMXEn8cHKK7MzZ2u617J52zIZ1y2Oj6d2Z9HuLPLb30TE4epzZVAjEXBqqCgUo91VL9Yui+kihsYfD/UW2A0BMO5j+PVR8Q+zxEgLTmUhlXYnZaXWkx7v6XA2v4sulRa1owI6jPdO1VSpYOwHEjVeI1gBbPpIRBb/CFSbP5UEsLVv+N55u6tFqKkhcwdk7YBNf0hS4tUfQWCsGFBv+ghV+/Folzwu4lqbKyG+F6T5qGDWmVC5naj+eBc1oNv1DcOaDuVw31fqtevV8M6KQ0wf2YaNyYUs3JmJzelmSOsIIvUNp1sB4k13PEGNpErrFGgTY+HX+/vz2doUDFq1l4hTl6//OMoN/RqfUlVJXrmtnohTw6I9WTwyvNXfUsgpKLex4oCP9j9ge2ox+eV2RchR+FtykVnoe6PX6+nWrRvLly8/tsztdrN8+XL69OlTb/vWrVuze/duduzYcexn9OjRDBkyhB07dvi8+BkMBgICArx+FP4aqYWVhFvOcvtD/iGwxJ50M51GRYhRdWFEkIc2FxNAXz3lCv/s76LGKMbdDRHcRFqc3A5odQWUZsoMaWWBiDt1cdrEw6AiD/DAhndh1jWw4D5IXimD0P4PysA9upPMTh9Pn3ul5WDZdPEoCG7sLeLUkLtfPHM8Htj4Xv31OXtg33y0pqATvnyH1kx2qZWHLmuFCg8aFXx1c08+ndadcd1i6RgXSNNw/7MrDisc4x/9XVQ4Y4T6GxjeLorPb+zBvDv78tzY9jSPsKDXaI5tk1IgrdgGrZqrOsfy/FXt+c/odvRrHorV4cbqdJ30eSptThwu77t4u8vNgh1Z7M4owajT4A5rjafTZFlZlinn1BpcVWLwHt+rdlmfe+CXB+tXSdpKYelT0PsuuPwVaUENbgzGQI6UQqXj5Md7OpzN72K5OgCPtUSEqFZ12sKaXSKG83VFnBo2vg+N+shEgs5PEqqOJ6yFxIcfqNPOFty4tuLJXgGLHhd/nubDoPMUmRRo3F/M8Q8ugkGP1hdM1FqZVNj4vtdiTfIKIvPW0zrKd4XQ/qwyNiQXsuJALq+M78i1vRJIL6ok23qScaruuDYlvT+u8V+QRwilVY4TPxbQazQ0j7Dw9Kg2PiPLa7A63FTZfX9u8sttZJdYKbPK8+WX2XxuB5Jj0NDzVDmcZJdUkVtmxe3DfPlCp8za8PsH4p2joPB35JzLsl26dPFpcqxSqTAajTRv3pwbbriBIUOGnNL+HnzwQaZNm0b37t3p2bMnb775JhUVFdx4440ATJ06ldjYWF588UWMRiPt27f3enxQUBBAveUKZ4/UwkoiLKfe73va2MtlJqzxqUXLh5tUpJVeABcurQFCmolpbM9bz/fRKFxIaDTQ7UbY9KG0WR1P9xvFcLMyH4Y+LQPlZkMlDWrwEzL4PbxcTCabDZEKGVsZfHqpCDsg1WBbPpN2rEnfinlkylppgXK7xPBSa5DHh7UQMSh3P/T/V3XUbAMYAvHs/JaGJBbVts+xdL4eT2x3VBlb6q13NhtOgSeApuFVpBdWctes7fx8b38SlOobBYW/BUEnMGFtFm6mZaQ/00e25cftGby2LBGDVs2oTjHc1K8JJr2mwcfWYDZoMWjV2JzeEzZfbTzK1D6NmNwrAXPZDlQhTavPlSug5XDYOEPOj1m7IGUddJxY3Yb6i7QWNdR2mp8o58+1b0hVisuOp9cdzDlo587Ykx/vhYLNrYKQ5pC2ATpfB71uhz0/SkXMj7c3/MDDy6Wq+LspYrp/y3LYO1+uTwm9RQD58fba1ipjkHgSFaXU7qMkTYSa1PXQdCB8fgXctFT+rSqSapxRb0HhEcjehTu0FapGvVGte9O7eqiaoJ2fMKn9m/wnu75hdesofzKLq9iRVkxaYSUzVhymzOakT0wUNzYejCZlVb3HuBP64QxpiWvYS+iyd+CI7oajyVCeWlnMykOraBcdwKMjWtMy0oLZcOJbLaNOS/PwBmtSMes1+B33Oc8vs7EqMbe6ms1O98bBPDisJbFBDXvgaNQqLMdV47jcHo4WVPDuisOsPJiL2aBlap9GjOkcS+Rp+PKcbwL9dKhU9U3Sa1CqcRT+rpzzipwRI0aQnJyM2WxmyJAhDBkyBH9/f5KSkujRowdZWVlceuml/PTTT6e0v4kTJ/K///2Pp59+ms6dO7Njxw4WL158zAA5NTWVrKyss/mSFE4Dj8dDakElkQFn0x/nsPwbePKKHBAhJ7XszM6S/WnCW0HaxvN9FAoXIkHxMPk78bmpQeeH6/JXoeiozGa6XbD8P9IWFRADV/xPbibm3yWzzE6rzFqW58Kql2pFnLoUJMmNy/JnJb1ly2fi+VBVIFGx824VDxxTCHSaDDqzPNfEr+UmaOLXYlwZ1UH2p9Wjsp3AgNNegc3ppmjkh7hju3utcjUayNG+z7Ex08klbSK5+oMNdIoPJCrw4hlgKigo/HniQ0z8Z3Q77vl2Gz9uz6Cwwk5WiZWPfk/m1SUHTyn9NNxi4LpeCT7XbUwuwM9ehGXFE7Dy+epzZYZUfUz4UsSE9e9A33tEoLCWQpcp9Q3jj8dlF9FHpRLBovO1NI4IOrtjnzNMiKcQ1aGlYuTsHylilqFacLCfoO3IViZiTVkWzLwCtCaI7gDdb5HK0fl3QmWhbBsYL21UK1+ovx+3U6pKIzvIBEVxKlz1nvxdRrwEyaskhtxegSsgHpXGIIb7E7+WZKvYrrX7spfj78M+TaWCaX2bcLQ6inzBzkwmV39W3liTx6E+L+FK6C8ba43Q/wE8UxfgHvo0uYUlLNcM5If4J/nCMZSObx3kp105lFY52ZBcyNj31rM+qQBPQ+pCHeJDTDQK9d06dVP/Jl7hIMWVdl74dT8Pz9lFUl4FJVUOlu/PZfS768ivsHF5+/rWEgBXdY4lzN/785eSX8God9Yyb3sGRZUO0ouqeOHXA9z19VZyS323el2IhPrruaJ9tM91fZuFEmpWQhAU/p6c84qc/Px8HnroIZ566imv5c899xxHjx5l6dKl/N///R/PPvssY8aMOaV93nPPPdxzzz0+161ateqEj/38889P6TkUzgxFlQ7KbE6izqbSn58oF9xT7FOOMKlYnXYBtFaBJFftXyDVDpbIk2+v8M9B5wdNh1A0dSVVhRngcVGkDmXWXit9Y7V0mjQIC5UEhESg2jlbZior8mSQ7HaCSgM6oxiA6y1SedMQB36GtqNh9cu1y3L2yL96fxGC5t4kVUItLpPZ0Xm31Uax+kfKQHvrTDzF6dD6SlS7vvP5VJ5ml7Axy0VoeCSH+rxPtLYcnb0YuyGYfSV6ggihdbSGX3dl0qdpCP+7plO9waiCgsLfk0A/HT/tyKS0qn7rxIHsMnanlxB9kiQeo07DnYObYXW4+W5LGq7q1pHujYN5bXwnDOVHxbAdRKA5tEx+klbAmPfwGIPA44IrXkW16iXQjxAfF7XGd4WkMUhMf+ffIYa4ncQbrH+zsAYrBi5ENLYSmRgY+bq0fqf8LpXOh5ZIvHjiYt8PbDKo1k8otBmoPHJ9UKmg6w1S8emohMAEqepc8kRt0mgNOpO8v00GS6t8oz4iANU1kW5/tVSE/ngHuvJc6HWnVPAcXS8TG8NfgL3zIHEJjpYjiY+JJdy/iLxymcAINul48oo2/LAljdsHNUOjVqHBxRWt/GkS1IbnFicyflYqjw96jksHaYgIMKL67f9QrX0DLRAHxCT0xXrle/R5/7jjr2b6/N10iO1/0smHyAAjX97Uk3u/3X7MdFmrVjGldwLT+jRGr62tyMkptTFve0a9fbjcHp6av5dPpnWntMrBuqQCoDZa/dERrbyqgypsTl5bdpAKH21bW1OLScwtP620rPOJxajj6VFt8eBh0Z7sY9+zIa3CeWFsB4IVIUfhb8o5F3K+//57tm7dWm/5pEmT6NatGx9//DGTJ0/m9ddfP9eHpnAOOFogsx6RZ3NGPS9RqnFUp1ZwFm5Sk1/lxOr0YNSeZ4+N8Nbyb8YWaD3y/B6LwgWHAzWPLStg2f4SVIDbI7PC31SfUiP8Day7XYvOVgb75sOeubJCpQY8Uncc1RHGfiitAQ3NKhsDof14byGnhv4PwPYvIbSFpLSUZsCiR723Kc+R0vlJ3+Ixh1Gu8icgqkN9TwWtEYY8SRN1DFanm4CoGGwuD3lWB0athhZBanRaFWqVipEdY5nat8kJ2zAUFBT+XhRX2vltX06D63/Yms7Q1hFoNSe+3odbjPx7ZBtuH9SU4koHZoOWULOeYLMel9oMGl1tq08N2btg1gTcV30IATFo5t8hXmW7vpdzX49bJVXpeAY8BNu+lIrHdW9JkIG1FHPT8RRVOogMuDjaq9xqHRp9dcWlrRRSN0qa1LKnJbEqeZVUedYlsr2ETJRmiHBzyX/ET6ftGElK3DpTfsZ9Cts+B3NEfREH5DqTsh6aDgK/EPjueig6Urve44bdP4iPTtur5Pgq80T06XytVPosegTGvAcZ2yluez2hRjPf3NKLUqsDh8tNUYWDT9YeYVtqEaWVlcwYEYhl3yw0a7bTMawVV99xM/utIVS4dWBx41nxCOrjotjVqevx++kWfrj2PYZ9erjey8gptVFcZT+lKtJGoWY+v7EHBeV2qhwuAv10hPsbMB3XmrU5pbDBfRzMKcPhcvPedV3Jr7BTbnUS6KcjzN9Qz+S4tLqSpyF+2p5B/+anNiF6IRAZYOTlcR15ZHgryqxO/A1aQv0NBCpJlgp/Y865kGM0Glm/fj3Nmzf3Wr5+/XqMRjnRud3uY/9X+HuRWigz9pFn0yMnP1EGE6dIZHUEeXqZm+bB53mAZQ6Xiom0TYqQ8w+i3Oogr9zOxqQCrE4XfZuFEmEx1ptFUqvApNfg8UhS6/HodWrsuiB0bUbBN3VMjj11Ks6yd8nscvebYcUzvg+o5WVSlTPxK2mrKkyGsFYyu1xwGDK3w1UfyMD+0FLf+3DZIW0j9hZX8uaWEh4Z+wWGnV+g3vE12MvxNLsEz+AnefJ3Gz/sWMO7k7uwdH8Ot/Rvws87M2ka7s9zv+xn1cODiT5B37+CgsLfF5VKheEEscv+Bi3qU2ivAvHKMRu0NAr1Xq6xhEP7CbDzGx8HoMYV2wvdwrsgYzP0uRtmTYSI1mLYG9pcxJzioyLY9K6uGkleWbuP1S+jmvAlYZRQoWrYC+VCw64PRtfrThGuWl4G/hHw68PiTZOxFSbNkqSq5JVSqdnleqnY+fF28cIZMh22fwV7foAr35J2221fVgtmHvn/oMfgyjfgjw8lbTS0hVxnYrtJK6/WJC1adUWcuuycJdciRwUsf0YqoY6sFh+3UW/jObCIyht+40iZmVCgzOZg/AcbvHbRKtLCiz2sBH0+6piYp05Zi2HrTBJGzeSuTaE8NziQqAO+LR9UGVtopsvn0UGRvLK6vuioUalIK6xkR1ox2SVWujYKJiHERLilfmVpiNng088lp9RKSn4FKfkVnEizVKnECyfQpCfwZJMeKjDo6ntH1XC8nw6Aw+kmp9TKjvRisoqtdEkIolGoifBTHNNX2p3kltrYlFJIudVJr6YhRAcaz5iHjcWow2JUhBuFfw7nXMi59957ueOOO9i6dSs9evQAYPPmzXzyySc8+eSTACxZsoTOnTuf60NTOAccLagkyE9Xz7jtjFGRJz3VQb774X0RXi3kpJZeAEKOSlXtk7Pp/B6HwjmjtMrBnK1pPPfLfq+y+2u6xfHY5a292ojyy+2MaB/F/B2ZPvd1Xa9GGAPCoDwA4npKbKwxEHIPyIC6PEfK/iuLpCz90OLaGczozhKnqjFIvEV0Z9npsGclASt1I6x9XUSdgQ9LLHnHibWeVL7I3oWxNItJbe6g/VsHeGDIRCZPvg6tRsXWbBee4hBmb9vC6E4xBJn09GoSwrsrDnOkoAKzUce4rnFKSbSCwj+YMH/xt3l1aaLP9df2SkD9V9PqdCYY8gSk/1F7PlOpoMlg7IOn4/G4UDXuD3n74eBiMYBf9ZIkM/W7HyZ8JZUpLruMQawl0gpbE09ddATs5RhMJvz8L57zmTEoElv7CRh+vgeK0mDcZzBrgpjotxkNwU2lQmf482Arh53fQlGy+NioteKjFt5aIrkX3CPiTN97JaEqu7pVd/XLIrp0uV4qf0rSYf1bkvh1aAW0vwqK0xo+SEcVGAPEHNleAT1ugfie8rcoz0OlN/PhH3m8s24PT1/ZlsahJlpFWjiYU2t6/PyloQQvHl+/IsvjJmTJvTw+ahF6Z2nDTrqAuiSN0U2a89Y6b2GkTbQFp9vDpa+v9lreMTaAD6d2P2lbIEB6USXTPttMUp5U0M68oQdqlVymj2dgi3CCT7FqNcxfz6QeCXz0e7LP9Vd3jfP63eF0s+VoETfM3OT1WtrFBPDJtJO/lnKbk193Z/H43F1exz6iXRTPXtXep7CloKBwYs652fH06dP5+OOP2bRpE/fddx/33XcfmzZt4uOPP+bf/5aYwjvuuIOFCxeeZE8KFyMp+RVn1wk/d7/8G1Q/Sr4hgo0qtGpIK7tAmtfDWkHWdnCdOE5R4e/B0YJKnv15f70x4pyt6aw/XHDsd4fLzRfrU0jKq2B0p5h6+2kXE8DYLjESwe0fKaXsa16HH++QFI+Rr8GAh2HcJ7DxXRkAD3wcrpsjQs9Pd4nRdkAU/PF+dST53ZC5TWZbV70gNyhhLSX1KnUDlKbLjHRDhDaHkjQCSw4Sajbwv+UpdJtxmE5vH+JfC1MptTqJDjQyvlsc/124ly7xwSzZl0OzcH+q7E7uGtIMo+7iaENQUFA482jUKsZ1i6NtdP3o6Ik94mkadobS64LiYdpCuHYO7j734L55Oa6EPui/m4Thw9540jdLS2pRCjiscNtqqZrNSxTRYtuX8M14MYN3WqVaJayl7Du4CeQdRK3ynJI584WCRq3C5heNZ9gzkL1DfG9uWizivt4M5dl4XDY89ioxxm9+iYg7ZdnSbvvhQGnvHfCQCDgZW+HXR+SaVNcDMP+QtGvNvQV++w9U5EvLVdNBUqEcFNfQIYpgpjOLqHbNF1CaCQvulQj47F14ulxHZoUKjwf+u3AfJr2Wxy5vhcWopVGoGAy3CXTI43xhK6WxoQy71l/EvYbwCyIw5Vc6xQcdW2QxaHllfCfu/HprvaqXXRmlvPXbIawniaMvszr4z4K9x0QckHbCR4e3rrdtmL+e/xvVloBTbCXSaTTc0LcxzcLrf4duHdCEuGBvYSa71MqNn2+q91r2Zpby2tJEqhwnHrNmFlfx6A+76glQi/dms2hP1imZQisoKHhzzityAK677jquu+66Btf7+Sll9H9XkvLLz27iTN5B6ac21B/0NYRapSLCpCK19AIyPHZUQe5eiO50vo9G4SzidLn5ckNKg+s/WJ1E/xahhJgN5JfZ+HLDUSrtTh4c1kpakfblYHO6GNginHaxAUQZHJCXJiadLoeUwO9fIDcZ+Qfh2u8lRaT5pZCxDcKawczLZdvQZrL8q6tqZx5LM2XmOXMn3LgU8vbJMq0RRr8j5fD97oP0zfUPXqOHpkNg43vQcpIITHWY1qcxJp2G1yZ0IqOoijsHN2NTSgEJIX7c2K8JMYFGpaVKQUGBqEA/PruxJ7vTivlhWzpmvZZreyXQNMxMyJk0Pg+IgYAYCgPaEPLLrWjSaltwVMkr4ehaSUzSGGD2tdBkoFTkfD1WKklq2DNXzsHjPhVxp9+/bSTFMgABAABJREFUwFGB0w0ql/ukfj4XCvllNgKrslF9PV781AY8JCmI+YcguBH4BaOylkL6ZjyhzVH5hcL+n8QXqIacPfDT3XDpf0ToObxcqqYNAWCJlrap4+lxi4hGB3+VytCRr0NIU6kGPZ4uUyVZ8ZKnJO68Jk3LaYPNH0PKGq7p/yk/bJfFP+3IZEgTI5vuaII79Q9cHvBTnXi8aNLCgWIDzdqMQbVvfv0N4rpD7gFMahdjOscQYNTSu2kow9tFcTivnJSCSp/7nbctg3uGNicu2HdaFUBBuZ3lB7x9bH7ZnYXJoOGTad35PTGPrBIrQ1pFMLBl2An35YuYID++vqUX21OLmb89g0CTjut6JtAo1FzPj25vZglWh+9x8k87Mrj/khbEhzR8Wzl3a3qD6z76PZkR7aIuGnNlBYULhfMi5Cj8M/F4PCTnVXBFh4Cz9yS5+06rGqeGCL8LSMgJbS5pDWmbFCHnb47D7Sb7BBGf+eU2HC4RVVxuD/2ah3JlR6m6cbrcNA03k5Jfyfurk3hjVAIcngO/v1wrxKhU0OdeGYA7bbDieSk573wtZO2EXbNry8m73wyrX/FdPp64CLpNg18eqjW3DGsphpcpa2WQ/vurtYPomtSq6gG9I6Ij2aVHAUnimNwzge6Ng/k9MY/CCgc70oo5kl/BXYOb8cLYDrSJtmDSK5cnBQUFISrAiK5RMM0j/VGrVISY9WfNC8NYmoI6bUP9FS4HJK2SdlWDBbpOhf0LvUWcGqqKRIgY9Q5U5ECjfmTbdIS5Lx4hR++xoln3mog4ai3E95KWWo0Ohv5bKj6rRXwVwLVzYMMM3ztb87qI/4eX18aZX/U+LH5MJuBAxj1dpknr1ZrXZFnWDihJhZFvwLLpx6VWjcfTZQqqzZ/A1i98RqKr8g7QxHaAhJBQUgsrGZygZVDhD+gX1LlOXv2xtIHVRKLXRWvEaY6kIEuF/ZJn0TutqOqmdSX0kevr3JupuGYO1zVrxITu8eiq/8ZL9mY3+P7aXe5j1/eGsDndPi/Jc7aks2BHJl/c2IPujUP+0mcqOtCP6A5+XNYuEjWqBlsVTzRWcbg8OFwNj6E9Hg8ZxVUNri8ot+NSKnIUFE6bczJSDgkJITExkbCwMIKDg09YWlpY2LAbu8LFTVGlg7LqVoqzgtMqZnmtR532Q8PN6gtHyNEaIKQZpG+Bnree76NROIv46bQMbR3BmkP5Ptf3ahKCpTqxQqWS2bPH5+6iwu4ixKxnWt/GtIsNYN3hfDrq0mH1S9478Hhg/dsyM6zzk+oYEPPjqA4ivtQQGC/l8Q2RuU1ExpoY8vxEWP+OzJgeXS+Gk3hkplWtgcVPQM4enP0fItPhz0fXdyfQT0uV3c2ve7K4YebmegPUgS3D6d4o+KK50VFQUDj72BwudmeU8NRPe9ifVYZKBYNbhvP0qLY0CTvz5sF+Kb81vNLgLwbvAx6Cw8vEO6whkldCQl9Y/gyeiV9xwN2IS+MunlZRs6cc9eFqM/vAuFrBpd1Y2LfAuxJTowNbCbgbaK+xlUoFzu1rxIdo9rUiDvW7T4Qbp128iowWqRKty6FlMgHRYQIM+be0+rqdcOBnVEUpeJoNRbXw/gZfR8SRn+ga/y8q7U76BeSi/7n6OhnTVaqwklfD0Kfh53/Ve2zJoGc4VOrH/y3Yhn5sB0YNeRb/7jdLfLrOCJk74IebsMYPwGqKJQCOiTgAXROCGzyuuGA/TCfxi7QYtQSbdBRVOuqtszndBJn0Z+x6qVWfeD+d4xt+LTGBxnrpWnVRqVRc2iaCn3f5qMACujcOxqxM3igonDbn5FvzxhtvYLFI6eKbb755Lp5S4QIkubrH91TM3f4U+YngdknJ72kSYVKxLt2Nx3OB9LCHtYK0P873USicA4a1jeTdFYcpqLB7Lddr1Nx7SQtMBi0lVQ5eXnyQBTtr+/gLK+y8sSyR2wc25cUrm6Df+EjDT7LrO2kB0JvBViZCi6NKBs6O6rJvlUpmOT0NCJp6CziPm1FLXiW+O3t/FIPKPveCtRiW/R9YorBP/I7fSuO46/O9aNUqPpranbwyG99trm9e2S4mgGbhZkXEUVBQ8CI5v4KJH23EVW2u4fHAyoN57M7YwE939yP2NNtJTooxsOF1LieYwuR8WVko59SG0PtDfrU4rrfQNdR0YYwvThG7C/z0/vI6HVW1r7XVSPjxNu+N3S5puT0RVYVQ6YF5t8j2IIJ/XcbMAL9g7+oYY6CYSKdukDAItRq+uErWFSRhHf4//PRmufb4wKEPpKrcw9RuYZg2PQ9xPWDQo+LZU3AYghMgvDWeGxbjXvcGmrz9uIKakNftX+T6NePFZZKYtS+rlOYRkUSq7YQnfoLf0RXgF0T+oBc47N+dJub6Ud3xIX50jA1gV0ZpvXXTR7Y5qWdkZICRR4a34skf99RbN7hlOBEB584gOC7Ij64JQWxLLa637t8j2xB1ktfSs0koUQHGepU9ahU8NqL1KXv7KCgo1HJOhJxp06b5/L/CP4vkvApUcNKT/Z8mZ69UHfhHnPZDI00qKp1QaPUQ6ncBDLTCW8GBhdWmf/UHBwp/H+KCTcy5ow/P/ryPVYl5eDzQMS6QZ8e0p3Go3KAUlNu8RJy6zN2Wzr96t0Xlf7eU+qvU4tFQt5e/PBcKU6DjZNj8kXyusndJqtW2L2WbI79Dy+FwcFH9J1FrJG63IKn+OpdDKsfcLijLoqrpZTgmL6DQoWVduot/z5cB6Ij2UWw9WkRJlYN3r+3Cy4sPkFZYhV6j5uqusdx/SYtTjjBVUFD4Z1BmdfD6soPHRJy65JfbWZ2Yz7W9Tj2l8mRY7U5sTYYTyH8b2MIjCUsV2ZC4GC55WipGfNHhGtjxDej9URkshBourtaR3cUGWnW4kcA1/5XEw4BoqRhWqaRVty4et4g95jC5vhxPaDMcGhM6e3GtiOOLinwwBHoLOW2ulMqX/EMSMd79pmOrStpP5T8rSni20034r3nW5y5zW05GvQ0md41EtdIokxrfT61NFQPQv41qwhfs6/kSh7OKKLRriHRHkHy0gv1Zpdw+sCk39W+CSa9hS4qHbzQ30Krr9ZQ7IKfUzHVtG/kUZcItRj6c2p13lh9m7rZ0bE43ccF+PHVlW/o0DW34fahGo1ZxeftoDFoNry45SHapFT+dhmt7JXDbwKZnLLb7VAizGJhxXVdmrDjMnK21r+XJK9rQr/nJX0tMkB/f3d6bF349wLJ92bg9kur1zJj2NI8485V1Cgr/BM5ZHVtpaX012hcBAWfRP0XhvHI4r5yIACN67Vmacc/eLSW6qtPff4RJHpNa6ibU7wKoCAivTiRI3wytLj/xtgoXDS63G5ebY9+Bmt+bhvvzzuQuVNiduN1g0Kq9TDzTCn2bJQb66Zg1IRbjr/dD0m8yVa0zQfcb4YpXJSEEILarVHj1vQeOrpGWqGHPyqA8daNUs+38VlI/snd7ez6oVDD8BfEgOB69WUrml06X3/2C2e3XD5spms1HCnl7hUT5xgX7cUPfxny98ShOtwe7081tA5oRYtbTItKfhBCTkk6loKBQjwqbk80pRQ2uX3Egh2u6x3m1s/wVSq1OliS5GT34OQJXTfdeGdZCDI5LM/BEdUZVli2CQ7urYe88722bDpbEpYLDMPYj3HYrav3FdbO6YFcWY5uPoFPMr2gzN8Omj+HyV6VFKiCmftLT+nfgitekWqeu0GMIIGvY+5R4QmmpKkPdkB8NiOl+XQPk/g/A3vmw6SOpcBr3MSyRhFtbyyspCO7Mjzv3M/X6K+gQ/SvarK1eu3P1uR99eFN6NKni1VWZvNDjFrTz7/AWcUB8gH5+kBZTf8UQ0BKzXoO/QUvXRsFM6hFHkEmPXivXqCGtI+gYF4jd5cbl9hDqr8dP1/DtVHSgH0+PastdQ5phd7rx02sarEx3uT243R50dcbJwWY9V3eNpV/zMKocLvQaNeGW2uM5l0QH+jH9yrbcMbgZDpcHk15zWkm0jULNvDahI0UVbXC5PfgbtYSdScNyBYV/GOdMyAkKCjphSWlNS4vLdeIoPoWLl0M5ZcQEnaUZd5dDosebX/qnHh5uks9mapmHLpEn2fhc4B8p6VtpmxQh529AYbmN5PwKvtp4lAqbk7uHNMfl9jDrj1RKrA5GdYyhbUwAM9cdIb/czmVtI2kSZiYywIBeq2lQ5Hh5eCTNf7sNVe7u2oWOSjGc7P8AtBwBR37H03UaquJUmHODzEa6HJC5FTpOkkSQwiQxLU5dL8lW6VsgZQ34R+DpfB0kLkF14Of6B9DjVtg959ivJb0eYsaWCu4coibAT8eYzjEMahmOXqtmzaF8JvdMYHNKEQ/P2XksgnTpvwYqIo6CgoJPtGo1oWYDxT48QgBiAv3QNmDO+mcwaNVsyXZSZOnLXTcvR7v7OzEubtRP/HF+vB10ZlSTvhZRZ9nT4tvS7io49BvghrZj5BybsQ0mfQt/vA/DngfNxeUBEhvkx/XfH+btK1+jXY8sIlN+Qm0Kl7FWv/th0WPeD8jehefQMly3raV896/4F+2lNLwb+VEDeGBJPkcLkvh6cis6DXoc1aJH6z2fJ743nsB4aHc1Hr8QPB3G4y7LRbt3Llz+Kq5GA/DsXwjxA8gf8gaHHBH8cVAEoynfpfLOqDdpq8kg4uhCPAYLtnYTmZ0Iz7y+DRCj/Rf6hvtOygIoPorRXkR0UCRphZXMWJlEdqmVS9pEMLhlODFBfqQXVbFsfw5rD+UTH2Li2p7xBB+X7uSLCpuTrGIrX208SpXDxdVdY+mSEHysQr2kysHRggq+3niUggo7I9pF0a95GDHVyY0qlersJr6eBkad5rTTserib9Dhb1DaqBQUzgTn7KqycuXKY//3eDxcccUVfPLJJ8TGxp6rQ1A4zxzKLadzfNDZ2XnBITHDC2nypx5u0qkI0HPhGB6rVNJelbbpfB+Jwl+ksMLG/5YeZNYm8YW5a3Az5mxJZ9am1GPbLN+fS+NQE/8Z3Y5bvtjCsn05tI8N4H/jO/Gv73YyuWd8vd5yo05NB0sZ6roiTl02fwpXf0xlv8cwHVoKK5+X5T/eIcaV0Z3E2ybvIKRvEvPj9K2w7m1JfgtvA3kH2ZjlJjR6FE26ZaLb+ZWYivsF4+lzDyq9Cda9CaZQins/yi+uXiTlFxFg1LLiQC5ju8Sy/nA+c7dn8M3Nvbjjq60UVdXekLWKtBDif/JBsIKCwj+TMIuBOwc15eEfdvlcP7lXwhn1nQk06bmuVwIPfr+TScGZROQfAr1JYreLJXmPgTeJt0ufu8Qk/vdXJJK8xTBofaVU4uz9Ua7fa18Djwf3pc9xAdT6nhbD20Xx6pKD3DIvnXCLgR+nPEycIwWW/ltSDke8KMmEZdliXNxmFM6+97M025+5qX3RafuTvrOKvYtS6BwXwKdjo4lRFeJoNBDtlW+gXvmCeN9o9Lg7TCSlw33cOiuTZhE3Qxl4lrtoHBpPq8ZPczC7lO8Wp9Kj8RAAtswr4uVxwaw4cAiACruLm+am0T4mgPemvEpGURVfrTlKeJ1qEZfHQ3G5jRM1q7udduZty+D/Fuw9tmzFgVxaRVp4a1JnJn60kZI617Av1qfw+oROXN4hGr8GJiQKym28tOgAc+rEby/bl0PbaAuf3tADf72Wrzak8L+licfWL9+fS0ygkdm39yEh5Ax7QCkoKPxtOGdCzqBBg7x+12g09O7dm6ZNm56rQ1A4j1gdLjKKqriiQ/TZeYKsXZIgYIn507uIMKk5WnKBCDkg7VW7Zou54kU2k6dQS0p+5TERx0+noVN8ELd/tbX+dgWVLNmbzSVtIlmyN5s9GaUsP5CLCpixMokXru7AQ9/vOJZeEW4x4Ffiw7OmBlspxfoo7lpaxcyusXgVL5eky0/2bplN3vMDFNcKSxQkQUESjtajWZli5bNNaYzpcA03jp1G02AN5W4DGnMoednpMG4FhQ4t726p4EBOPp9M687XG1PxACa9lvk7M3l5XAcW7c32EnEiAwy8P6WrUlatoKBwQga3jmBsl1h+3J5xbJlaBc9e1Z644DMfnpAQYmJK7wSsocGw9O76G4Q0larHOTdIgtPYj2S5swq2fg5+QaDS1AYWBCVg11qoqnJguYgMXaMDjbw9uQv/mr2DvDIbuzPLiAu2iWH+ujfFNHjodDHCV6nh8G8UJ23mmz3N+NelLbjtq60UVzoY2SaY/3QqJnzpNKgskJ03vwzHpO8otGnItarZkq8lrMxMWlEVSfm1MeK9moRwRYdoHpu7C5fbw/L9uQBM6B6P1eFmQItwDmSXARBk0vHMVe2psrl4dckB9meVseCe/hzKKWV9UiEeD+QTSJhGLxN/x6Mz4TCG8p+Fe+utGtE+iid/3OMl4tTw2NxddG8c0qDgkpxX4SXi1LAvq4yftmcwtE2kl4hTQ2aJlTeXJfL81e1P2LqloKDwz0U5MyicEw7llONBXO/PClm7IKiJpBn8SSLMKo5eKBU5ABFtpI87Zw/EdD7fR6PwJ/B4PHxbp/KmZ5MQ1iTmNbj9wp1Z/Hd0O5bszQZgzpY0JvVM4KVFB3jx1/08d1UHSq0OMouraB8TgMXvBKdwjZ70SjU70kqouLIvhuAmUHTEe5uqYqnEiWgLufu81+n88Ax6gsEVEYQGhxDgpyPVrWNPuoNmEf6YKkEXnEBGsZX9BaVc3yeC5hH+pBZUMr5bHDani/SiSn6+tz8Wg5aWERb6Nw8jrbCSlpEWWkT4E322zgcKCgp/G8L8DTx9ZVvuGNSUjcmFmPQaejQOIcJiOGHk8Z8lKtCPKzvEgLUIV4dJaHbP9t6gIk+qFvMOwq7v5acufe6BrJ3Hfi0c9AJHK8xE65wXlZBj0GroEBPAN7f0YldGCbkeFe4gDWqNTlrH0jd7R5ADpmvH8cTlrXG73XxzSy8O55YxNLQYy8wrvU2ODy9Fd3gp/tOWsbAojKaRZiodLj6Y0pWUgkqKq+x0iA0i0E/LvG3pfDK1O3szS3F5PHSIDWR9Uj4PfL+Dn+/tT7BJR+MwMy0i/CmtsjN/RzZPXN6WyEADUQFG3p7clcziKrYeLaJSb8Q18HE0K5+p/4IveYrNeTo8Pjyp28YE8NbyQz7fJ4fLw77MEp9Cjsvt5us/jjb4Hm9LLcaHj/cxFu7K5KHhrYgNUm7XFBQU6qOcGRSOYXW42JdVSuNQMyHmM9vucCBbzK7jz0aJqMsOefug+bC/tJtIk4p1GReQR1NocylXTtukCDkXKR4PVNidx37XalRYnQ2LhTanC52mtk3A6nCjrzbxPJRbzt2zthFuMTCgeSg9GgfjCGyMzj9SEkWO31e7CWCOYO4djdhVaiPm8m+JOzgT055vwWWDFsOh2w3wy0Nw6f/B4eXid2OvwN3iMtyDp7Mk00RSQQGjO8VytLCSlxcf4FBuOS0j/LljcDM0Kvh2UxohZj1L9+ZQWGHHpNfw2IhWLN2XQ7voAPwNWmKDTWc+IlhBQeEfQ7BZT7BZT6uovxaI4XZ7yCm1UlBhx+PxEOpvIMJiQHucWbJGrWL0Z3v5+abp+Ed1w7zpbSjLwhPThaLIPgT2j0bz4631n0CtheaXiDlvXHfcAx9jcXYsjS1un8lbFzo2p4f3VyVxpKACs15Dv9gmNOswAdWOb+pvbInCE9QYW6WbIJOOT9cewaj2MOLozAaTqgzrX6dt15f4aMNRVlVPcgxvF8mE7vH46TTsySjlq42pfPNHKk3C/NGo4Z3lh3BWv5dZxVbC/PV4PB7Si6qotLvo1zyMhFDTMRPeMH8NYf4GOsYFyZNG3wBhzaTduDAJwlriGfIUFVE90BSq+WBKN6ocLr5Yn8KOtGKAk/7tquy+X5/bA5UNrKvZ74nWO1wePL6UJQUFBQXOs5BzJvuaFf4aG5IKuPMbKYM1atVMv7ItU3o3OmP7T8wpIzLAeHZMTXP3yexQaPO/tJtIs5rcSidWpwej9gL4bGr08prSNkKv28730Sj8CdRqFeO6xvHrbqmw2ZFazJMj2/CDjzJrgMGtIthUJ6HlsnaRBJl0vDmxMyVVDuZvzyA6yEifZmHcP3sHraMtfDJxLuY5E6G0tu3A3ewS3IOeYMbiDBbvyebTaT0Y9VUKN/SewsOTx6ErPQpH18P314O9Ar79Q/wdLnsWV1xvZu+3U5looEuChfRSByPfWUtkgIGJPeJpEu5P83B/Hvp+BzvTS+q9Bj+dhogAI1N6NSLEpCf6LLQ+KCgoKJwuVoeLTUcKefD7HeSXS2tNoJ+Ol8Z1YGCLcMx1qnvUahXXdE+gUBXIs0e68fDYn8gtqUSt9yM0MIq95QF06XEv/ltmSPQ2gMECV74h59Sr3of8RNR75mCIfQw/nQY//cU1d6pWq/DTqxnaOoL/W7iXb27uxR9Z5Tja3k/L0iw0yStqNw6MI3/MLMZ/kUxKQSVGrZpHR7RiZEt/DD5alWrQFhzEY6tkSOsIViXmcVnbSC5rG8WjP+yi1Org/SndABFEkvLKvR7bOspCUl45A1qE8dOOTD5fn4KteqKkaZiZ96d0pWWkpf69hilUzKkb9QWXA4dKyx85au55Z/sxU+1Qs54nr2hDhMXA0n055JfbaBxqIqXAd4Jkxwb8H3UaNeO7xbFsX/3JFoDGYSYuaRPBuysP+1zfp2kIFuPFU8WloKBwbjlnV5Wrr77a63er1codd9yB2Wz2Wj5v3nERjgpnnZT8Cm7+YjNNw8w8NKwVqxNzmT5/D346DeO6xZ2R5ziQXUb82bqhy9opMciWvxY3FVmTXFXqpmXIBZKiE95a4qEVLlraxwbSMTaAXRmlFFTYsdpddE0IZluqd6SuSa/h+t6NuPNr8c8JMeu5vncjnvppD9uOFhNuMTCpRzzD2kUy6p21OFweDudWkGVoT8mwOUSri1BX5mO3JKAPiuTFpTksqhaQFu3OYkrvRvywPYMHmoNu3nEzyR43JC6B1I0cHLOYlgnRXPPBBj68vhvf/JFKlcNFSkElLy8+CMCIdpGM6xbnU8i5Z2hzkvMqcLhcjGh3ljyxFBQUFE6TtMJKbvx8s1d1RUmVg7u+2caCu/vRIS6I/HIby/fn8Nm6FN6a2Jmd6cU0j7KQbPXnpRXpvD6hE0/+uIcNyQXc0nMU9958HQH5O1CpNaA1wMb3IXXDsf1X9Lyfdglh+Ol0Z7zS+WyTXWpFp1bRp1kodw9uRpXDxb/n78WgVTN96FMM6jkdfVka/iFRHKyy8MAPuaQWitBhdbp55uf96Ea15LqI9qgbCG5whbYgJjwEe5maVlEWJvdM4JYvtxz7G21PLebKjtH8vMs7aUqjVnHv0BYcyCxhW2oxH/6e7LU+Ob+CiR9t5Od7+zecsOQfAUBiZgnXf7bWq6WqoMLOIz/s5NNpPVidmMesP1L575h23Dhzc71WqGl9GhF+Aq+3TnFBtI22sC+rzGt5oJ+OqX0a42/QMrR1BCsO5HqtN2jVPHVlOwIvonY8BQWFc8s5M9EPDAz0+pkyZQoxMTH1liuce/49fzcWo5YHh7WieYQ/N/VrwsCWYUyfv4e0Qt+zD6fL/qzSs9NWBZC1A0KaidneXyDCLELOheWT01YqLUoyTr6twgVJZICRj6b24InLWxMX7Md7qw7z2IhWTB/ZhkahJkLNesZ1jeWbW3oxc90RTHotU3onMOvWXtz+9VY2Jhdid7nJKK7itWWJvPXbIe4aLNVnz45pzz2ztpOvDmVWejg3rQ9h/NwCijwWFtYZ+M7Zmk6QScd/x7Sj2NwULFE+j7Wi251kugPZn1lKm2gL7644zDXd4+ttt3hvDjGBRj68vhvdGwUTbNLRNSGId6/tQu8mIcQH+zGkVeRZ8a9QUFBQOF3sThefr0/x2SLj8YihfEG5jdeXHuSxubsx6zUs3ptNqL+eIa0iiLQYefLy1mjUajYki2HvJ5vy+WivihJDLCy8H76f6iXioNGh7TaF7zenU1Rlp7jSh8HuBUx+mY1Sm4tym5OxXWP5qFossTndPLU0i8FfZDNysZk/7I0Z93XKMRGnLi8vTcLR7ZYGx2d5Xe4jpUzFw3N28tLVHfhlV6bX3+j9VYfp3TSU6SPb0DLSnxCznqGtI5h5Q3dQwVVdY5nRQDVLcaXjWGtUQ1Tanby/KsmnL47bAwt2ZvLwZa14c2JnuiYEs+Ce/gxtHUGwSUfrKAvvTO7CvZe0IOAEYktUoJHPbujBYyNkDBDmr2dK7wQW3NOPhBATof4GXrq6Ay+M7UDTMLE2GNUxmp/v7U+LCP8THr+CgsI/m3M2yp45c+a5eiqF02D94XzWHS7goWEt8dNLFYpKpeLGvk3Ym1HK87/s44Pru/+l58gttZJfbqdxqPnkG58u9nLIPwxtx/zlXQUbVBg0F5qQ00b+TdsIgePO77Eo/GmiAo3cOqAp47rGYnd5UKmgSZgJrUaFVq1mR1oxb/6WyIh20dw6oCkxQUZu+WILyXm16R09m4QwtFUEWo2Koa3D6RIfSJjFyA19GzNnSxoVdifju8Wj06gpszrrzRq+tjSRa7rFcSDAwKTR3xO35BbIr07KUGtwdLmRrWGjsbvV7MksISbIjw1JBUwL8t1iuXhPNvdf2oIrO0YzoEUYeq0GnVqFQa8h2HRxzTwrKCicHYor7didbswGrVfr0rmmyuFmf1Zpg+sTc8ooqXIcSxiMCfIjMaeM9jGBlFbZCQ8w8ti83XwytRuPjWiFVqPmj+QCAow65mVYGDvwGYLX/J+0VQGYw6ka8zEfbrfjUanILbURFXBxtZkWVNiJDjCyP6uMdjEB+Bs0x177xuQCVh7Ixen2kFPasEClVqk4YA0hdvRXhC27FyoLZYUhgKIhL/HhPh09W7kpqLCzZG82ibne7VNuD0yfv4dWkRam9mlEj8Yh8nCtmgiLkZwyKxO6x7NgZyZH6qRd1bA3o5R+zcJQqSDIx3Wp0u46lnrli6S8cp68vPWxGPP2sYG8NakzFTYnOo2a0FNMXYwK9OO2gTIGcHsg2KzDoK2t/I4IMHJtrwSGtY3E5XYTYNQpEyEKCgonRTlL/MP5eE0yjUNNdGsU7LXcqNMwsUc8761KYkdaMZ0b6P89FfZmyuCpSdhZqMjJ3iNtIX/RHwdEwIoyq0i5kCLI/YIhIFbaq9orQs7FTKnVwYHsMt5ecYi0wiraRAdwXa8EftqRyYKdmdw7tDk6rYrXliVyx6BmHK4WcQL8tLw6vhO70kv4fksaDreb/HIbl3eI5vlf9pFWVMWIdlEMOFZFV8Uv9/ZHpaLeLOPO9GLGd4tjry0I28jvMDmKUDutYArBagiFMjB5oE10AD9sTadJmJmsEqvP19MyysLDc3bxx5FCVjw06OxV3CkoKFx0FFbY2ZFaxDsrDpNdaqVzfBD3XdKCJmHms+OVdxL8dBpaRlrYllrsc32zCDNFFbWCRFaJlcEtw2ge4c/SfdlU2Fw8d1UHNiYX8cPWdGxON0NaRdCrSQhfbSxjQ0U7nr9+NWZHIRVOSLWaWXTAw6XtothytIgQsx6D9pwVwZ8RAo1a1HjoEBuASa+lY1wQc7akY3fJa595Qw/+b8FewvwbFu6tTjcujZHb1gfy+PAFxOrKwOMh1x3AGxtLWZucT9+2jQF5zxuHmtnlo2X3YE4Zaw/lgwcW783m3yPb8uT83fyRXEiYv4Eb+jbG7fHwzM/7vK57scF+3PbVFhwuD/cObU6XhCBCzLXii0mnoXm4P4ePE5BqaBlpIcDkXW1jMer+lG+NRq0ioloQaohwy6kJQwoKCgqgCDn/aNIKK1l1MI9bBzb1aTzdr1kYC3Zm8uZviXx+Y88//Tx7MkrwN2gJO8WZi9Mia4cY15lCzsjuIk3qC0vIAanKObrufB+Fwl+g0u5kzpZ0nv91/7FlWSVWVh7M5aWrO9I41ITd5ebhObsAcHs8x4SY58a0541liV6zhh+sTubH7Rm8PK4jN32+mU/WHuGnHZm8NqETN36+meIqB8PaRLL0OIPFxJxyBrYI5/8W7OWPI4V11hQT4JfKu5O7klZYQa8mIfx34T7evbYLry45WO/1+Ok0NI+w8MKvBxjSKpzQEwzkFRQU/lmUWR18uuYIM1bVtrxklWSzdF8OX9/ckz7Nws75Mem1am7s15jvt6T5jHu+Z0gLyqyOY79vPVrE/8Z35F/f7WBHWjGfTuvO68sSj01MAXy18SiL92Tz1S09+WpDCu9tt/P5+vzqtfLvzA1yXo2wGAi+yDxygkx6VCqwujzc++129mXVf+3/u6YjqYWVtI6y+Kxsubx9FC63hyqnhwnfptZZUwzAqI7R/F6dVrVkbzbvXdeNhbsyfbY6jesWh8vt4a7BzRnz7jrsLhmrZZVY2Z1RwlWdY7lzUDPeW5UEQIBRS4TFwObqAIGbv9jCnYOactfg5sdi4E0GLXcNacbivdn1nk+lglv6N/GqnFFQUFC4kLi4pgcUzijzt2dg0Knp0zTU53q1WsXoTjGsOpjHwROUnp6M3RklNA41nZ2UssztENrsjO0u8kKryAGIaAc5+6Cq+HwficKfJL/czitLDtRb7vHA/5YeZET7KD5ec+TY8i0pRfRvHkazcH/yy+0+B8g5pTZWJ+YxqKUYNuaV21i6N5tL20RQZnVw99DmDGrhfcM0pFUEOaXW40QcobTKyfdb0jiYXUZ+uZ37LmlOTKAfV3WJPRaBDhAZYODNSZ15Z/khBrQI44WxHQj0u7huUBQUFM4e+eV23ltd37fE5fbwxLzd5Jb5rvI72ySEmPjo+u4E+NXOYfobtLwxsTPNws00DjNj1Mm5rnNcIIk5ZexIK6ZRqImM4iovEaeGvHIbszelcVO/JnyxIaXeercHXl5y4KKMHjfoVOSV2dieWuQl4tSQV25j+f5cWkVaeHZMezrEevtcXtY2kruHNKfS4eCZ0e3pFOe9fljbSMZ3i+O7zdLOZnW4mbctnXcmdfH6G1kMWv47uh1L9uaQUVTFi4sOHBNx6jJ/RwbdGgWj06iIC/bjzUldeH1Zotc2H/yeTH6FdytY03B/3p7UGf86rUwBflo+mNKNhFCl0lRBQeHCRanI+Yfi8XiYtz2DHo1CTljm3KdZKN9tSeOTNcm8ek2nP/U821KL6Hs2ZuAq86EkHZoMOGO7jDSryaxwYnd50GsugAhygMh2gAfS/oCWw8/30Sj8CdIKK3G4fA/k88psVNpdxwb6AUYtfxzJ59Hhrfn9UF69JIu6LN+fy7W9Elh7OI/YID+ySq2MaBeFXqsmq7iKDvFBTOvXhEq7Ez+dBpvTzfwdmV77CPTTEWrWU2l3kpRXwV2Dm+Gn1zCifRTrDxeQkl/BJ9O646fXYNCq8TdoqbK7ePWaToT56336DigoKPxz2ZdZ6rOiAiCloJKSSgcRlhO3mJwN/PRaBrcKZ/H9A8kvt+F2ewi3GAgPMKDXaNBr1Xx4fTee/Xk/0/o1Pnau7NkkhJUH8hrc75K92YzpHHPsNTcNMxMb5MeBnFLyyuwcLaikpMrR4OMvVNSoSM4rZ/mJrkEHcpnUM560oipevLoDlXYneWU2/PRaLAYNWcWV3P71Nu4e0pznx3bA6fZQaXPip9ewM60Yk0GLRq0Cl+xv5YFcru4SywtXdUBTPQbzeODrjUdZn1TA+1O6sjujfutVDbllNpb8ayCbUwp5ZuHeenHhfjoNyXnlNAmr9Wz0N2i5vEM03RqFkF9uAyDMYiDSYkCrUea7FRQULlwUIecfyqHcco7kVzD+JPHiWrWaYW0imbctg8cvb33Kxm41ZJaI0fFZcd7P3Cn/hpy5ipxoswq3RyLImwdfIOW0lmhpHzu6ThFyLlK0JxEFNWoVMYFGHrqsFRq1ivSiKoqrHEzp1Yj/LNjX4ON0GjWtoyzMuqU3eq2aQzllOD0eQs0GcsusvL8qCZe7tk1raOuIYy2O8SF+PDisFXqNinCLAbvLQ1JuOSa9BrNew7pD+VTYXVzZMZqWURaiTtLbr6CgoACgO4Xz3flCq1ETE+RHTJAYDxdW2EjKrWB3egmBJh2NQ828OLY9jcPMrDwo4o3b7UGvbfiYdRo1LreH4W0jueeSFhzJK6ewws6/hrVApVLx2A+7zutr/rN4VCr8jTqviszj0WlUrD1cwIuL9h8TsmquN+9e24UAoxarw81rSxN5bWkiNYXZNdsue2AgC+7px+rEPHZllNAxNhC9Vs1NX2zx+XzqBiq7B7cM5/o+jSi1Oth0pJBwi4HRnWOYsVKugQFGLY8Mb02ov56M4ip+T8yjWYQ/MYFGVCoVOo2a2GA/YoNrDaldbg8ZRVUcyi0ju9RK2+gAYgL9CLvIfWysDhd5ZTb2ZJRQbnPSKS6IiACDMimjoHARogg5/1CW7MnGT6epVwrriyGtI5i7LZ3vt6Rz5+DTE012VBsLNj8bQk7WDgiIAf2ZS8OK8pdBwpGSC0jIUamkKidF8cm5WIkN8sOk11Bpd9Vb1zjURKCfjpfGdeSJebvJKK46tq5rQhB3DWnOioO1M6Kd4gK5oV8TTNXpUKFmHWU2mQUtszn56PdknryiNU3D/RnWJpLFe7OPDZo3HSnklfEdWXsoj+fGtOfZX/bx9JXteGLeHpLyas0ew/0NfHZDD5JyyxjQPAyjXrlUKCgonBqtowPQaVQ+qxDbxwYQZDp9o9izQW6plcfm7WZlnYoTo07NW5O6sPT3JO4c2IxRHaNxe0T4BhVLfHipjO4UQ7i/nnHd4hj//npsztq2n8Gtwnnvuq7HWrYuJtR4aBZm5qb+jbm6axwePGhUKoqrHHyyJpnEnHJGdYph+f4crwosj+f/2bvv8Cir7IHj3+ktM+m9QwgktNB7ExQbihUUBVnbqljWn7uru5bV3VV33bWsvTdQUbGgIKIoIr33koSQ3nuf/vvjkoSQCRJISAbu53l4NPO+8+bOJJO5c+6554gAT3yQCaVCgU6tbH5Ojj2vT4gPa9JKOFBQw0UDRFHo/MoGJiUGY9CoaLC3fb/MKK5lfEIga9PLmm+b3j+MSYlB3LFwe6stV9P7h/H0lQN59Ot9vDB7CP9ZeajV9jg/o4aFN4+if4SlzdZ/p8vNnrwq5r69iepGR/PtKdF+vDpnKOF+3tWBrEmDzcHqQyXc88mOVq/PmSkRPHxJstcHqSTpXON97yxSp/hhfxGDo33RnETaqEWvYUyvQBZuzOrwPu8tmeWEmLsi0u+G/J2dmo0DogW5XkXPq5MTOkAErqyeOytIPVuIWcez1w7m+MVEnVrJv64ehNsN/1x2oFUQB2BXbhVRfgYmJQYDMC0phHlj4/jHt/u5/cNtXPv6Bq54dT17cqv5bm8BizZl8/SVA8GtYOnOfG6f1IvgY7Loaq0O0opr+ccVA3hs6T6m9w/nldXprYI4IGof/H7hNlJi/WUQR5KkDgk263jyioFtbvfRqfn3VYNadQ3qLg6niw83ZrUK4oCo0/LKz2lM7x/OrR9u5dYPtnH7h9u48pX1JIebuW9an1bnJ4b6MDjaF6vTzZ2LtrcK4gCsPlTCF9vzuqZGYBezOd38dKiYtKJa7v90J3cs3M5tH27j2ZWp/OH8RK4aEsmMQREeawc9cEFfXlyVTr3NwUMXJ7V57zNoVPzpwr68tz6Tz7flcrikjuGx/gT6aHly+QEevqTtffQaJX3DzdwxOaE5GKhUwOwR0Tz81d42dXO+31dIflUj905N4KPN2W3GWVlvZ+47myn00JmxsKqxTRAHYGdOJf9ZeYh6m6PNfbxBflUjd320vU2Q9aud+Xy3twB3e3siJUnqkeQM/RxUUmNld14Vd0w6+SDI+clhrPl6L7+kFnNev9CTvt+mI2X0CzOfyjBPrCoXGso7pe34sRQKBeE+CjJ6WiAnbCC4HJCzERKmdfdopA7SqlVMTAzm+3sn8u76I2SV1dM3zMyVQ6NotDvIrqjnUFHbgsYj4wP4YkceU5NCmDE4nCh/I3Pe2tQqoFrd4OCRr/fyxo3D+OFAEQ98tpsPbh6Jw+3mL1/s4emrBrI3r4rNmeX4GbQkBPvgZ9CSWVZPSrQfL//ctigpQF5lA9VeWNdBkqTuZdCouGhAGP0jfPlgQybZ5fWM6RXIZSkRRPn3jOKxJTVW3l+f6fHYPVMTmffOZuqOyaBstLt47sc0XpidwtwxMWSW1jNzSCRDYvyorLPzy6ESHO0sdH2yJZsrhkZSUmP1qvbSlfU24gJN3LFoe6vbC6sbue+TnXx793jWpBbz9rzh/JpWwvbsSsJ89Vw0IJwfDxSxYl8hvYJM6DRKFt48iq935pFb0UBSuIWJfYJ4/sc0csrF4sWa1GKuGxnDXR/tAMCkU/P2vBGs2FtAbkUDfcPMTE4M5n8/pVNZb+e5a1PYmlVBg83Blsxyj53IABZvzuaNucP5t4fuiwDldaKG0fEZNmnFNW2COE2W7srnvmmJGAO87yPUN7vy232uXvslg+n9w36zRbokST2H9/0Vkk7bL0dbPQ6O9jvp+/QONtEr2MT767NOOpBT1WDnYEENEycGn8owT6xgFyhV4B/X6ZcONSnJqOxhgRxLFBgC4MivMpDjpYxaNYlhZuKDTBjUKiL9DKiVCvRqFTqVkjsn9WZAlC9KBdidbvQaFYEmLd/syqfB5mRgpC+fbc1tNyvuky05XDYogoWbssksrcOkVXGgsIZ/rzjEHZN7MzI+AIfLzVc7cpnQR3S6snvo/HGsiuO6e0iSJJ0MH72G5AgNf585AJvDhcvlpqrRTn5lA2a9utvrcThcbo8f1JPDLRwoqGkVxDnW/1al8ddLkrD2cmLUaWiwOYnwN5C3SxRGHh7rz9XDozDrNNTbHHyxPY8NGWU4XW6crh42r/gNbuD1NRkej1kdLr7dnc/QaH8ySmsZlxDI7BHRVNbbyatsYEiMH4eLa7E6nfSLMBNs1nLlkEjcQIPNyQcbs9iVW9l8vfEJwc1twwG+3V3Ayn1F/G5cHBP6BPPZ1hzeX5/ZHISY/94WRsT5MzMlkm1ZFe0+huIaKwoF7QYvAIpqGnG53BTVNGJzuNCplSd8b7Q73Vgdnn8/OlNZrZU6qxOVEgJ9dCdsTHKyso4r/nys4ppGr+yuJknnMhnIOQetSS2hV7AJX8PJ71NXKBScnxTKG2syyCqrIzbwt+vSbDhcihsxMep0+TvBLwbUnT8ZjDAp+DW369+kO0ShENurjvzS3SORTlO0v5F+YRY+2ZLNk98dJNrfwNNXDSK9pJbX1hzG5YZR8QHcPqk327PKmZAYzL9XHKS4xsrBwrYp7E0ySmoZFusPQHpxLVH+Bv4wrQ8hFj3P/ZhKVlk9Jq2Kq4ZFkRxhQadWtqlfcLzIHrJ6LkmSd9KolORWNPD0dwf5YX8hLjeM7hXAYzP60yfEp1u6AjmcLhSIBarDJXWtjkX46dtsNT1WRmkdEb4GHv5qL1uzKugfYeG1G4YyKj6AKH8DVoco7FtSYyXQpOWG0bHMHhmNTq3E6GXbVI0aNRkneC72F9QwoU8wv24rJdgnihX7inh19WHK6mwEm3XcOiGe8QlBrE0rpc7q5JXV6eSUN2DWqblmeBT/mz2EP3y6E7vTTZ9QH9KLW38vm9PF4q05PBJmZk1aaZvvvyWzAq1ayWWDI/hiR57HMQ6K8sXlEu3EqxvaBu4UChgY4ct7GzJ5+ad0MXYfHXdN6c0jlybx928PtLlPoEmLqQt/lvU2B3vzqvjb0v3sL6hGp1ZyxZBI7pnap7lI96malBjMl+0+V34YtD2kNqQkSSdF1sg5x7hcbtamlzIg4reLHB9vbO8gfHRqPtiQdVLn/5IqWiJ3epqm2wmFuzu9Pk6TcB8lJQ1uamw9bGUifLDIRGqo7O6RSKchKdzMHz/fxfI9hThdbh6d0Z+7Fm1n5f6i5lXDTUfKuXPRNkb3DuKORds4WFhDYVVjq5apx4sLMjXv9Y8LMqFTK4kOMPLQF3uaV+HqbE4+2JDFE0v38+y1g/l2dz43jI71eL0LkkMJ8pFdLCRJOnW5FfVc89p6vt9X2Pz3bWNGOVe8su6E2QFdpaCqged/TCO7vM5j84aCqkZ6B5/g72ygiQ0ZZWw9mgWSW9FAg91FcoSZoupGnvn+ECU1ooV1WZ2NF1alsSOrApNWjaUDi2c9gUalIO4E7znxQSb++0Mql6dEYjGo+ceyA5QdzeIsqbHy5PKDbD5Sjk6j4qEv9jRvo6qxOnhnXSafb89lwZQElApRV6mXh+e9ot6O0+Vu92dy8YBwQsx6QtrZsnbbxN68+ethbhnfy+PxB85P5MONWTzxzf6Wsdda+ds3+ymobOSywRFt7nPvtD6EduH2owMF1cx6YyP7C8TCjdXh4pMtOcx7dzNFHur5dMSI+ACP2/sUCvjLxUndniknSVLHyEDOOeZgYQ3ldbaT6lZ1PK1ayZR+ISzekkNN44lrZ7jdbn4+WMLAqI5/n99UdhhsdRDg+Y35dEX4iJdFj9teFT4Y3C7IXNvdI5FOw8aMcoqqxUR/WKw/O7IrmieQx2q0u3h77REu6h8OwI8Hirh0UATtdbGdPSKGpbvyCTRpSQjxoV+4hX+tOOjx3LWHS4nyNxLupyc+yMidk3tj0YsVRp1aydwxsfx95gA5qZMk6ZS53W5W7iuitNbz37fX1xym0UNnoq5SWNXI/He38MP+Ivbm17Azu5J/zhxAuK/4UK5UQKhFzwXJYRjbyUy4fWIvPtmc0/z11cOi+HBDJrVWZ7uLXB9uyqbG6n3FcQ+X1DF3jOdAv1al5KIBYWzMKOPVn9PZmVPp8Txfg4b/rUrzeGz1oRJSYvx4b/4IjpTUMm9MnMfz/rvyEK/dMIyLBoQ1t3EPteh4bEYyGaV1vPxzOu/OH8H4hMDmAskxAUZev3EYy/cUsHRXAcPj/Pnj9ET8jxZJ1qqUzEyJZEq/ED7Y6Pnn9sGGLK4fFYNGJS4aaNLy5wv7ogAqu6h+XHmdjSe+2Y+nmsNpRbUea+l1RKSfgU9vH8OEPkHNt8UEGHnvphEkdUU9S0mSupR35XlKp2394VK0KiWJoaf2B3t6/zCW7Sngk8053Dqx/UDKzpxKCqsbGRbjf6pDbV/BLlBpwTe6868NRBxtQX640sngkB6UZmoOE+3WM1ZD0qXdPRrpFNgcLn44UNT8dXK4ha2Z7e/v35JZztwxcXy1M+/oqlw2/75qEP9YfoDKejGR9NGp+b8LEvk1rYSLBoRxw+hYVEqobrQ3B4w82ZZVzuj4AEDB8Fj/5sLLBo2aCD89Bi/bBiBJUs9Sb3O2+nt3vHXpZVQ32Dul9sfJ2JNXxcHCGmamRLI1s5yV+4sYGOnLvdP6YNFrUCkVrD9cxtPf7WfhzaO4c9F2CqtFBoROreSOyb2psTpafZhODrfw+i+HuXRQRLtbVJ0uN2W1VhJCfM7I4+wsy3bnE2TW8YfzE3l1dTqNdvH4gn1EEKWmwY7bDTVWJ8NjAxiXEMimjPJWRZ+1apXHhYomhVWNbMus4JYJ8aiUCh66qB/P/pDa/FyGmHU8N2sIxdWNRPkbeGXOUNxuMOvVzYG3XkEmnC43ExODuWNyAgEmDQEmLd/uKmDp0dpFFXV21qWX8cilyeg1KhQK+OlAMZll9e3WhbE5XdidLl68bigut5sGu5NPt+Sw6Ug55/ULAVPnL3Q02Bzsyq1q9/gvqSVMTDy9upPxQSZevn4o5fU2HE43Fr1aFjiWJC8lZ+rnmHXppfQNM6NVn1oyVoBJy/iEIN74NYMbx8S2OwH7dncBfkZN19THKdgFAfGg6ppfX71aQZBBQXpFD8vIAQgbDIdXdfcopBMorGpgd24Vy/YUEOij5eph0QQatRTVNLInr4rAYyZ/dTZHqzaqw2L9sRg0pBXVkl1ej59RQ90xK7nf7yuivM7Gp7eNobLBjtPlar7e4ChfvtiRxxtrMrhkYDhxQSaUJyjyGGDS8dnWXFbsK8KoVfHUlQNZuDGLt+YNl0EcSZJOm0alPOH2TD+jBrXqzLTldrncfL1T1AapsznwP5ptuCevigeX7Gk+b3SvABZMSeBAQRVPXjEAo06NSqHAz6ghq6yOf61o6X4UcDT7cd64uHY7VjXxxtojIRY9r6w+zPnJobwwewhOlxu1UkFNo4NFm7KYNSKGV28YSn5lA++sO0KfEDO/GxfP0l35fL1TBFA0KgUKBR4zTABUSgXf7y9iRkoEe3OrmNIvhGGx/tQ0OjBqVfgZNdQ02NmYVcmbvx7hzV+PtLmGr0HD/2anYNKqCTHr+PlgEbvzqhnbO5Bv7x7P3R9vx+V2sz27gvWHy1rd94L+J27eYdCqqKy38/g3+5ozy3x0atQqBTnl9aw+VHx0G7OJFXsLqGqwc+ngCAZE+hJ2CsERpVKBWaduN4Mr1NI5Xc8sBo3XbfWTJKktOVs/hzicLjYfKedSD3t+O+LywRH8mlbCok3Z3Dw+vs1xq8PJlzvyGB0fiLK9fSCnymmDor2QcH7nXvc4ESYF6T1taxVAxBBI/Q4qMrukY5d0evIqG7jxrU1klLYU0DRq1dQ0OFiTVsLDFyfRa0gUn27NBcR2qSevGIgbmDU8mo0ZZZTWWrl5fDyBPlqcTjf/+6l1WvqIuAAWb83h7bViQnvbxF7YHC7eO6aV7tJd+TxwQSLnJ4Xy/f62K+IGjYq+YWb+77NdAMxMieS7vYXcPqk3vga5nUqSpNOnVSuZPy6eb3YVeDx++6TeBJjOTDtupVLRvH3017QSXrxuKJ9uzWl1Tv8ICzeMjuXm97e2yq4xaVW8O38kRp2amyfE8+CSPWhVSp69djD3Ld7JkdI6Hr00mYSQtgV7AaL8DVj03veh+dJB4byy+jA/7C/ih+PeR+6dmkCEn567P97RKvPz/Q2ZPHJJMo12F9/vKySjpJYJfYJYk9q2WLFZJwIvL8xOIbO0nsOldVj3FTIoyo9Xfk5na1YFgSYtr984jNG9AtsNCF09LIp312WyOrUEP6OG52el8NbaTL7dXUCgScu7N43g483ZXDY4ss3PvKTGSnSAobl+z7F6B/uwJ7eK99dn8t9rBnPbh9uwOlzcPaU35XV2Zr2+gVkjYnC4XPzpmGDgVzvz6Rdm5t35Iwj37Vhx4iAfHXPHxPLyMR28migUMC3p5LrGSpJ0bpA1cs4he/KqqLM56X+aWTLhfgYmJYbw4k9pHtsTL99TQHmdrWvecIr3g9MOgQmdf+1jRJiVpFb0sM5VAOEpou162g/dPRLpODaHi9d+OdwqiOOjU9M/wsLyPQU8NiOZPy3Zzcr9hdw1Rfz+Vjc4qGt0MK1fCDe/v4XX12SwZHsejy3dx5PLDtA/0kKwT8sHHaNWxaAov+Ygjl6jZHisf6sgTpNXVx/mzikJ9DquWKVOreSl64ewcq8otjww0pfzkkIIMGoYEu3X+U+MJEnnrF5BJu6b2qfN7TNTIhjbK/CMjmX2iBhA1Of5Na2Ee48b1y0TevHo1/vabJESReKPYLW7MGpUXDQgjAsHhLFsdwFHjv69f/PXDB6+JKlVxiWIbJGnrhjItqzyLnxkXcPPoObRS5Ob6840GRUfwMQ+Ibyy+nCb7btuN/xz+QFmj4gm0KRlaKw/t0/sTUxA6w6Ieo2S/103BKVCwT2f7OCRr/fy+bZcnv8xjTsWbuPmCfEkh1soq7Px1HcHya2o55FL2o5lWKw/I+IC+CWtBIDKejv/XZnK3KP1dsrqbPxz+X6iAgxM7x9K/4jW8983fz3Ci7OHNGfGNgk0aXnk0iTeXJNBZlk9X+3M56IB4QyP9eeSQRHc8/EObE4XI+L8eXddZpvn7mBhDQs3ZJ+wjbknGpWSuWPiGBHbuiyBUgHPX5tCmK/cAiVJUguZkXMO2ZBRhkGjIv4EHRlO1rXDo9h0pIx/Lt/Pf65Jab7d7nTxwo9ppET7Eel/em0SPcrfBTozmLt2VSLKrGRlpoNGhxu9+sykfp8UrRFC+otAzshbu3s00jHKaq18dtxq37iEQH7cX8SNY2J57odUyupsvLsuk9kjonn3phFsz66gT6iZa1/f0GYLVH5VI08uP8iC8xKYU2/nUGEN/SPMrEktaT5ndK9Afj5U7HE8dTYn93+6k7fmDSezrI4tmeVE+hkZ3SuQslorLuC9+SMwalX4GjT86cJ+srixJEmdys+o5Xfj47l0cAQ/HyrG5nAxpW8wYb4GArqgxsiJRAcauX1iL15fk8EHG7K4ZngU7940gt25VTjdLiJ89ZS3U8/l0kGRLPhoO1aHi8dmJDOqVyAzXmxpPFBQ1cjflu7j7zMHUFFnI7einugAI8FmHVaHk9d+yWBcQhBhHczQ6E5Wh5s9eZUsu3sCP+wvpNHuZFLfEJwuNy7c/HLMe9GxnC43BVUNPDdrMHmVDfjo1PznmsGU1VrZX1BNkI+OpHAzfgYNdyza0aYtuNXh4uEv9/Lwpcn8YfFOtmVVcNvEXmzbX8w780aQWlRDvc1Jn1AfSmtt3Ld4R6tMnT15Vdx9Xsti36YjFfz1kmTWp5fyv9lDKKm1simjjBCznrEJgYRadCy7ewKbjpSxL7+a+CATASYtj3+zn/yjXaKW7yngm7vH42/UUFFvJ624lkmJwe0+BwCLNmdx45iYDv/MQ331vHLDULLLG1ibVkKgj47xCUGEWHRe18JekqSuJf8inEM2HC6jb5gPauXpJ2L5GbXcODqW19dkMCTGnzmjRGeDF1elkV1ez+8ndU1rcPK3i25Viq5NJos2K3C54UiVi6TAHra3PXIY7PoY7A2g8Z5J4dnODc3FIJvo1CrqbE4GRfny7A+pzbd/siWHT7fmMKVvCEE+unbrK6w+VMxfLupHsFlH72AfAowalu0pbHX9elv7mWO5FQ0cKKgmzFdHoFHLoEgLRo2ShF6BjDrDq+GSJJ2bmupxdHexX3+jljsm9WbG4Ag+25pDdaMDP4Oay1PCqai3o1Qo+OslSbyz9ggFx7R5tujV1Fkd1B39W/vI1/t4dc7QNpk7mWX13LloOxG+euaNjWNjRjlLd+Xz2g3DsNpdOJwnrqPT0zjdbgoqG/lhfyHf7ytCr1Hy4cZsovwNPHhRv3br3gA02J3kVTRSVN3IoGg/lArwNWrQqZV8sCGT2EATt0/sxeGStlvRQGTSGDSq5jpvDqebZXsKWL63gKHR/jx55QCueW0D1Y2ea8nYj3uu621OrhgSRaivnt4hPow+7v0v1KKgqMbKjuxKvttT0BzAaWJ1uDBqVQSb9c3t5XVq5Qnff+utzhM+RycSbNYTbNYzLLYLGoZIknTWkIGcc4Td6WJLZjlXpER22jUnJQaTWVbPX7/cy9bMChTAFzvymDUimtjA08/6aaOxSrQeH3h151/7OJFmEShKLe+BgZzokbDtXcj4Bfpe2N2jkY4y6dSM7R3YqpjizpxK/nJxP3x0bf/UutyQXV5PZX37HT1cbiivt3PTu5uptzl58KJ+nJ8cyre7Rc2J3bmV3DutT3NhyeNNTAxmS2Y54xKCeXPtEWKDTJh03lerQZIkqTP4mbT4mbQMiPSlst7Gt7sL+Nd3B5uLy/YLM/PkFQP5x7IDzUEGEZBvHTDIKK1jdK8ANma03TKVX9VIuK+eV38RdU4a7U6mJgUTbD4z9YA6i69Bw18vSeKLHXnsL6huvv1IaR2BRws9e6oJBBAXaMJHp+a7vQU8f7T9uEal4Nrh0bx8/VAq6m0oj98ndZxGuxO1UhTMrrGKLo1uN+zIqTjaLctzECfYrGvV1j7K30C91YFO43kBsLLexvI9BUT7G9ie7bmL5Kh4/+b3cT+jFj+jhl25lfzfBX35ckeex/tM6ReM2SA/ZkmS1HVkjZxzxO7cKhrtLpIjfDvtmgqFgnljYrlpbBybjpSxIaOM+WPjuPw0iym3K38H4IbAtvvtO5tJoyDQoOBQeQ+sk2OJAkskHFrW3SORjtE06dWqWv6s5lc2EOVvJKeivk3tBBCBnOSI9mtW9Qoysb+gmnqbk8RQH8w6NU6Xm35hZkCk86uVijb7/kHUILhhVAx+Bi1f7chjSt8QQi16jF7YPUWSJKmzbT5SzsNf7W0VEDhYWMMfPt3Jny/s23xbWZ2VwVF+re4bH2Ti95N6t/p732RYrD9VDQ4q6+2olaLj1Q1j4tCdoTbrnUWlUPD80a3yx/rduHg+2pzNvVP74KmfxfT+YUT5G/j7sv2sSWspcmx3ulm0KZuFG7P4eHMOxTVWj4scAGqlAotBg83p4r7zE/lwQ1bzsfEJQaQX13Lt8GiP971vah8Wbmw5/5FLk4nwM7S7dXhdeil/+XIvpbU2RsS1zYDRqBQ8cmn/5vuHWvQ8NiOZomorCmj3/ffeaX3wkQsnkiR1IRnIOUdsbKqPE9S5mTIKhYLp/cP47zUpPHttChf0D0PxG6sspyxvK1giQG/umusfJ8as4GBPDOQoFBA9Cg4uA1cPHN85rE+oD0vvHse0pBD0GiUzUyL4bFsOizZmc7eHgp9Whwubw8XUfiFtjikUcO8xE9LZI2J4a+0R/vbNPh6Y3pdbJ8QTaNLyr+8O8eiMZBacl0CQjxaDRsUFyaG8M28E9TYHCgX8klrCvLFxFFU1EuJlq8KSJEmdrbTGyr9WHPR4rLLeTm5FA0nhZvpHWHj9xmHUWu3MHiECB4EmLQ02J6//ksEbc4cxpW8weo2SELOOOyb15pYJ8fxz+X4A5o2NIzbAiAo3tUezSrxFUY2VVQeLqWl0tCqEPyI+gI835/Dt7vyjHaUC0KmVRPkbuP/8RO4/vw8VdTb25lV7vO5n23K5aGAYn23N4Y7Jnrfh3zA6lr15VbwyZygbM0TtGgCtSsnd5/VBo1IyLNafhy9JIj7IhE6tZEi0H+/OH0FOeT37C6oZHuvPJ7eNpnewqd25b3FNI//+XrSUf2r5AW4aG8+dk3sTatGh1yiZ0jeYpQvGkxjasi1QpVQwtV8oi24ZxZLtufxhWiK3TezV/P57fnIor98wjNUHSrA5emD3U0mSzhoy5+8cIerjmFF1djvwM8XthNytEDn8jH3LaIuSzQU9NFASOw72fQFZ6yF+QnePRjpKq1LRL8zCc7NSqKy343C5eOCz3WzLqqBfuIXXbhjG/1alcaCwmghfA3NGxVDdYGfG4AiSIiws3pJDaa2VIdF+/N8Ffflkc3Zz6nqoRd/cIeXWD7YyqU8wD17Uj0h/A243nN8vlEl9gvA3abE6XGw+UsbHm3NICPZh4c0jqWt0MLJXICoPK8iSJEnnEpvTxeGSunaPpxXX8r/ZQ1ifXsb/VqUzPiGQ2yf1IiHEh3XppRwuqWVDRhn78qu4algUt0/qhValYtGmLO5cdJhofyMLzktgeKw/V7+2nhevG4pRp/GqDI3iGlEn5snlB3huVgqbj5SzZHsuDUfrwny/r4itmRVcMzyKa4dHU93o4OsdeSQeLULcHqvDhdsNq1NL6Bdu5umrBvL2r0dIL6kl2t/IHZN7Myo+gJpGBz/sL+LHA0WolArG9g5k/rh4XvvlMAvOS8CgVRHlr+eJy/sTE2A82uLdRVygkVkjo9GrVZh0aiwG8ZxX1Ntwudz4GTTN74N2h4ussvrmcd310XZG9wpgwZQ+GHUqdColSR46vVoMGpLDzVw0IJzqRjuDIn3pH2HB4XKzI7uSOxdtJy7IxHWjY9CqZRMBSZK6xlkRyHn55Zd55plnKCwsZPDgwbz44ouMHDnS47lvvvkmH3zwAXv37gVg2LBhPPnkk+2efzawOVxszSrnyiFR3T2UU1dyCKw1ENz3t8/tJDFmJUvTHVRZ3fjqelgALCgRfEJg35cykNMDmfUazHoNNoeTuCAT27IqWLQpi6uGRvDnC/tS0+igvN7Gkm257MqtAmBkfAB/mNaH4bEBLNtTQEW9jbXpLWnplfU2wix6CqsbmyfBq4/pmPHABX1Ztief9OJaLkgOY2pSCH+c3pekcAsGjYpAH5mJI0mSBGLrToSvvk1R2yaJoT7YnS42ZZbhcrsZ0zuInPIGMktrWTAlgeJaUfC2utHBu+syeXddJr2CTFw7QtSACffT8/CXewny0VJWZ8fXoEHvZVurAkziPaPe5uT3C7cxISGIhy5KIuqYjqRldTZe+yWj1f20auUJO5KplArURxcVX/slg15BJl6YnYLLLearH2zMpKrBzr9XHGRy3xAeuSQZvUbFrtxK7v90J5X1dq4bGcMrqw9zxZBITDoVSWEWGh1OVu4rYvGWHNy4uWZYNBcNDMPqcLI2vZR312XSYHNyyaBwrh4aRVSAEZVKZFIV17S0Ud+YUd5c++jpKwe2+zh0ahXrD5fy/b4ij8fjg0zo26nLI0mS1Bm8/i/M4sWLuf/++3nsscfYvn07gwcPZvr06RQXe27Ju3r1aq677jp+/vlnNmzYQHR0NBdccAF5eZ6LlZ0NdudWHq2P034tjh4ve4NoO+7reU90V4jxFS+P/WU9MCtHoYDY8SKQ4/SudO1zRXWjnW1ZFVw+OAKFAu6Y1JtvdxVQb3ey4OMdPPr1vuYgDoh6DU+vOEhqUQ0vrErj483ZXH+0GxzA59tymTsm1tO3Qq9Rkhxh4UBBDfaj3T3u/3QXDpebKH+jDOJIkiQdI8Si5x4P211BdCOKCTCyZHsu901N5InL+nPnou289HM6QWY9DQ4XCcE++BpaZ9dklNbx9HcH2ZZVwUs/peN2Q0ZJHQMiLfge7dzlTULMOvqGiq3sbjesSSvlz0t282taKUNjPHdTGhHnz57cKuqtDmICjB7PubB/WKu23RmldbiBWW9s4K21GQyJ9ifK34DLDT8dLOaPn+/m7o938NavR6ist2PRq6m1OtieXcEjX+/lp4PFWB1Obnp3M48t3cf+gmoOFNTwxLf7ueGtTaQW1fKHxbvYnVtFWnEtz/+YxhWvrie7vJ5Qs65Vq/JjGTQqogOMFFV7DvYZdWrumOz5vgC3T+yNQXNWrJdLktRDeX0g59lnn+XWW29l/vz5JCcn89prr2E0GnnnnXc8nr9o0SLuvPNOUlJS6NevH2+99RYul4tVq1ad4ZGfORsOl2HSqojvik5SZ4RbbCEK7ged0Dr9ZEWYFGiVsK+0BwZyAHpNhoZySD97f3e9ldPl5ueDxVz35iaWbM/l75f3Z1isP2+vy8RXryHIx/Nq5S3j4ymrs6JUwLr0MmICjFw5JBKFAnbkVGLSqbluRHSrApOBJi3v3jSCd9a1rIqqlQrum9aHUfGyxbgkSZIn5yeHcvP4+FZ/TwNMWp6blcKrqw/zzrpM8qsaqLU5MGpVbD5STohZh1oBb6w5zLvzRxDuq291zUsGhjMg0pf8igYeuTSJVQeKeWH2EGqtDqyOHjqXaIfb7eKRS5OagzlNtmWV8/RVAxkU2bp5xuAoX+6aksDba49QXGPl2WsH0+u42jTjE4K4PCWCz7flNt82LNafrNJ6Lhscyff7igj306NWKghuZwHiupExrTo1fru7gJyKBnLKG9qce7ikjq1ZFW0KNpfUWHn71wxsDhcXDghnzqiYNu+rz81K4T/fH2JPXhXt6R1s4t9XD2qVeaPXKHn22sHEB3kOZEmSJHUWrw4V22w2tm3bxkMPPdR8m1KpZNq0aWzYsOGkrlFfX4/dbicgIKCrhtnt1qaXkhRuQemt9XHKD0NNISRedEa/rUqpIMaiZF9pDy1WF9BL/Nv1kWxD3sMUVTfy+Dei2OXXO/Px1WuorBeZU3/9ai8vXT+UB5fsprLBjkappLzexuzh0YxLCOJgQTVvzRtBRZ0Ng1bFnNExzBoRTW5FA2a9momJQaJwcXUjBq2aIB8tP+4vZNbwGOaMisWsUxMTYCLIR4uxnY4gkiRJ57pAHx3zxsQyPiGI8nobBo0Kq8PFyz+nNxfXTS2qIbWwlutGxvDCqjT++tVevr5rHJ9tyyPApOPl64dSVmfF5YaYAAN2pwu3y83Lc4ZS3WDn95N64XaJ7Un1Vgc6tfdsryqssnLf4p3cNSWBKH8jjXYn0QEG1qWXcdM7m1lwXgJPRAzgSEktZoOGjJI6/rB4J5elRDC5bzCpRbXcMbk3fkYNjXYXvYJM7M2v4p5PdmA9WgR4eKw/C85L4P8+3cVjM5JZtjufvXlVRPvr+ejWkdzzyU4OFNQAoFTAVUOj6B3iwxu/tt7O9f2+QobF+JNaXEN1g4OGY9qPr9xXyJR+IezMqWx1n6W78rljcgIKheg6+fa8EZQffd9ttDt5/sdUDhbW8PHmbCb0CfL4szPrNVw+OIIxvQLJr2xAoYAIPwMhZh1aL/pZS5Lknbx6ll9aWorT6SQ0NLTV7aGhoRw86LkbwfH+/Oc/ExERwbRp0zwet1qtWK0te2erqz1X4e+pGu1OtmdXcP3ImO4eyqnLWANaEwR67m7QleJ8lewt6cGraL2nwrb3oLYEfIK7ezRdypteixX1NsrrWoo97sipZGKi+PkcKa3j8205vHjdEAqqGrE6XPQLM+N2uzlYWEOwWU9lvY304lpG9QrgQEENSgWE+xr4fl8B8UEm/rXiILtzq6izOnh2VgpPfie6bkT66fniznGEWvQexyVJncGbXouSdCJOt5v5723BoFFhd7pwuNytjvvoNEzsG0RCsJnduVX8fKgYh1Oc8/qaDHbnVXHXpN6E+ekprbGi16hwumFvXhVBPjrK6myENmftdP5iWle+Ft1Aaa2Nx7/Zj1IBkX4G7pycwDNHuzz95cu9RPuL23z0biL89Cy8ZRQK4LKX1zU/T0oFTEsKJdxXT4SfgU9uG0Od1YFBq0KlULAvv0p0lgr24amrBuF0uugV7ENlvZ0nLhNtv+usoq7cTweKeXDJbtzH/JjMOjVT+oYwKTGYI6V1BPnosDlcvLAqjezyepQKBa7jfq5iXApcbjc2u4ulu/J5ZfVhj78HKoUCxQl+drqjW7Ci29lKVl5no6TGSnpxDYEmHTGBRsIseu9dXJUkqcfw6kDO6Xr66af55JNPWL16NXq95w8+Tz31FI8//vgZHlnn2ZJZjt3ppn+E72+f3BO5nHD4ZwgbCMozv7oR76vkp2wH9XY3Rk0PfNPtfR5sfx92fAgT7u/u0XQpb3kt1jba27QcPVBQzX3TRD2GmSmRDIj04+rXNjSvSioVok1tkI+OZ74/xB+nJ2LQqrj5/a04j04oNSoFf7k4CbVSwbasCqobHQDNRSMBZg6JIvAERSYlqTN4y2tRkn6Ln0HL4CjfVvXKmigVEOGn56Z3t6BTK3no4iQCTBp0GiWj4wPYeKSc2gY7Rp2aW97fyuOX9efxb/c3Z5AAxAQYeePGYRg1Sky6zp/DdOVr0degIdCkpazOhssNORUNRBzd9tQU6MipaOChL/cA0DfUh2uGRzO2dyB6tYpap3iPcrlhXXopL143BLVKyd0fbye3oqE5GDM9OZS/XJzE/Hc3t+p2NSo+gJvHx/PAZ7v477UprDpQzKJN2a3GaNCoeOG6ITz3Yyo7siubbw/31fP0lQN5+Ou9XDwwjG93F7R5fFcNjSS1sIb31h/hwgFhvPXrkVaZPE3mjI5Bqz61bf1F1Y08uGQPPx9qqdvpb9Tw/vyRDIj0lcEcSZJOi1fXyAkKCkKlUlFU1LpifFFREWFhYSe873/+8x+efvppVq5cyaBBg9o976GHHqKqqqr5X05OTqeM/UxZm16Kv1HTqsuAV8nfJurAnMG248fq7a/E5Ya9PbVOjs4McRNgy1twdNJ0tvKW12J2eQNbsypa1RVwuNysP1zGTWNimTE4nMe/2dccxAEx0X13XSYBJi0p0X5E+Bl59ofU5iAOgN3p5vFv9pNZXs/NE+IBkZbetH8/OsDA9aNiUMv24lIX85bXoiT9Fn+TlqeuHIRR2zbIcsfkhOYAgNXh4vFv9nH9qFgq6qw8MiMZs07NwzOS+d37W5gxOIJXVh9uFcQByC6v566PdmDUqbtkq01XvhaDzFr+ecUAVMcEGxZvzeHhS5LanKtTK7n/gr7sya2iuLqRv8/sj+KYGEWdzYlRp+JvS/eRU97QKqPm0sERzH9vS5uW5ZuOlPPd3kKGxwVw3+Kd3DA6ts1CxeyR0Xy0KatVEAegoKqRh77Yw0MXJTEoyrd5q1yTKH8Ds0bEcPMHW/klrZQxvQKJC2ybUTMpMZjk8FNbCLU5XLz2y+FWQRyAino7c97aREFV25o+kiRJHeHVM36tVsuwYcNaFSpuKlw8ZsyYdu/373//m7///e+sWLGC4cNPHCDQ6XRYLJZW/7zJr6ml9I/wRaHw0qj/gWXgGwmWiG759lE+CnQq2FXcQwM5AEmXQXUeHFja3SPpUt7wWrQ7Xby3/gjvrj3C01cNxKJvSXp8e+0RhsX6s+pgcatJ7LE+2ZzNHZN68enW9ifjS7blMqlPMIEmLQ9M78vevGr+fnl/Ft82hkg/Lw3YSl7FG16LknSy+oaZWXbPeG6dEM+QaD8uSA7lpeuHYHO4WhXldbvF31+tWs36tBK+umsctY0OKuvtDIryY9ORco/XP1xSS0W9naqGzu8w2ZWvxXqri19SS3hr7nCuGhpJSrQfWpWKUfEBfHzrKC4cEEZKtB9zRsXw5tzhLN6czXWjYtBr1fx0oJg35w7nssERpET7cc2wKLQqFRmlda2+h59Rg9XhpKLe83OzbHcB05JDya1ooLimkTfnDuPOyb0ZEu3H5MRgrkiJZNVBz11q86samztvvT1vOOMSAhka48/jl/Xnk9tG8976Izhdbtxu+NPnu/nzRf344/S+DIv1Z0zvQF67YSjPXDOIYPOpdX0srmnk483ZHo/VWB3sL5BbUiVJOj1ev7Xq/vvvZ968eQwfPpyRI0fy/PPPU1dXx/z58wGYO3cukZGRPPXUUwD861//4tFHH+Wjjz4iLi6OwsJCAHx8fPDx8em2x9EVSmut7C+o5o5JZ762TKeoyoG8rTDwGuimQJRKqSDeV8nOnhzICewN4Smw9jnof0W3PVcS2BxOaq0O/ji9L35GDc/PHsLevCoOFlYT5W+kf6Qvnx7zweB4+VWNmPUaCqs8tzsFyK9sRKtS8vWCcQSYNLwwO8Xr2tpKkiT1FCqlgvggH+6e2ofyWisfbMjiL1/uobqhbZZrdnk9jXYn723IZkisPwWV4m+13XnipgjVDXasdid40d9qu9PFF9vz+GJ7HtOSQhkRF0BRdSMzX1nPjMHh3De1D0dK6zhSWsuBgmrmjY0jq6yOwqpGvtldwA8HirggOYwRcQEU1zRw5LggDoCfQUNxtdXDdxdsTlfzwkdFnZ2qejujewUyoU8QhwprKKhqbHdhBER9moFRviSHW3j6ykHoNSrMehUlNTamJYVh0Kr5bGsOZXU27li4nYGRvoyMD8CsVzMiLoDAdjpnnQybw0Wjvf3fC09dtiRJkjrC6wM5s2bNoqSkhEcffZTCwkJSUlJYsWJFcwHk7OxslMe0rH711Vex2WxcffXVra7z2GOP8be//e1MDr3LrU0rBWBQlJfWx9m9GPS+ENb+1rczIcFfydbCHhzIARhwNfzwMKT/CH3O7+7RnLOUCiX3n5/I22uPUG938urqw/gbtcQEGNmVU8nY3gEMj/Xn16OvzeP1j7CQV9lA/wiLx0lv0zn+Jg3hfrK1qSRJUmex6DUoFZBTUe8xiAMwMNIXg0ZFaa2VBpuLxDCxAKhRKVvVjjlekI8OHy/rImjUqugfYWF7diXL9rSuMfP5tjz6hPgwtncQy3YXkF5Sy8HCGkb3CiDx6LbixqNFhJvMTIlq8z2Kqq3EHdei/Fh+Rk1zzbn4IBOPLd3LzpwqLAY1Vw+NYkR8AAaNymNtGxAdpJ5ctp/FW3NptLsY3zuIBVMTeHFVGuszyhjTK5D/XpvCKz+nszWrgj15VezJq2LmkAiMp1nTyKBVEeyjo6TWc6BqQKTMZJQk6fR49daqJgsWLCArKwur1cqmTZsYNWpU87HVq1fz3nvvNX+dmZmJ2+1u8+9sC+IArEktIT7IiJ/RC4uflh+BjF+g1yRQde/kp2+AksI6N3k1PbQNOUD4YAjuB6uf4oTLU1KXyq+s5/YPt/Px5hw+2pTNbRN7sSevimV7CsirbKDO6iQl2s/jhF6pgBtGx/Kf7w9x9bCoVkWMm+g1Sq4YEomvN76mJUmSejgfnYb7pibiqQatSatiev9QtmaVMyjKl8MltQSatPQLM/PD/iKuHNo2UAEwLSkEi16N0csCOYE+Ov44va/HY/5GDb2CzXy6NReVSsHBQlEb6GBhDcNi/T3eZ3t2BZMSW3fXbLA7Kau1kRzuOahx09g4lmzPZUyvANaml7IzR9SEq25w8M66TL7bU8C8sbEe7zsqPoCd2RW8vyG7OTNm7eFS5r69mVsn9sJHq2b94TLuWLiNu89LwHS0TpJaqeDOSQkYNKf38wo167n/gkSPx3oHm4gNbD+AJUmSdDLOikCO1JbT5ebnQ8UMivLr7qF0nNsFm14DUxBEjezu0dDHX7y5by3qwVk5CgUMvh7ytkHq9909mnNSdaOdHTmVHC6pBWBffjWFVY08dFE//I0a5oyK4eWfD/PMykO8MDul1cQ1yt/Af68dzOfbcimptbInt4r3fzeS2GOKLyaG+vDW3BFE+xswar3rA4EkSZK3CPTR8vzslFZNIpLDLbx0/VD8jBo+2JDF7ZN6szuvkka7i5euH0JFvZWR8f7cODoW3dEORxqVgquHRjJvTFxXdB4/IwJ8tLx03RBCLS1bjIZE+/G/64bw9HcH+GhzNpcOCuemMeJxV9bbqbM6GRLt1+Zab609wqOXJnPV0MjmhQq9Rkl+ZT2vzhnKtKSQ5gCaWadmwXkJGLUq/I0a7picwIs/pbW55ks/H2bG4AgWTEloLlitUiq4fHAET1zenye/O9jmPjaniw83ZnHF0EhAFLJetCmbK4ZEEhdoZNEto1q9954qpVLB9P5h/O2yZCwG8Z6tUMB5fYN5/3cjCbV47pYrSZJ0shRut1y+74jq6mp8fX2pqqrq0QUet2dXcOUr6/nbjP70DTP/9h16koPfwsZXYcTNEJjQ3aMB4P9+buC8GDX/mNCDi8m63fD9Q+L/b/8VlGd3nLanvRZLaqz8c9kBvtqZB4BWpWR8n0CuHxlDv3AL1Q12/r7sABqlguIaKzNTIokPNqFVKwm36NGplVQ02NEoFVj0akpqrKBQoFEpUSkVmPVqwnz16Lqg84kknY6e9lqUpNOxLr2Uv3+7n/nj4oj2N2LSqXEDpTVWDuRXcvGgSFbsK8Cs0xAXaKRPqJmc8jrMBi1qpQKFAuptTqob7CzdVcCXO3JZce9Eeod0fR3GznwtVtbbuPWDrfgZNdwxKQGtWonL5aai3obL7caNgkCTluyyOqoabYyIC6KgqgGH042fUcMP+4v4ZEsO1Y12RsUH8MfpfQny0RFo0lJUbaXW6sCkU2HUqjBq1ThdbsrrbDQ6nBi1apQKUCkUaNUqHvxiF5uPVFBva7ugdv/5fTBoVET4GVEAPno1SeFmXv4pnfc2ZBHhqycx1EyN1cGO7ApcbrFt7InL+/PAZ7sBETj68q6xWAwaQsydG2BxOF0UVVupsdrRq1UE+mgx672nVpIkST2XXNY9S/10oBizTk2fMzBx6FTlGaKVdszoHhPEAUgKULEhvwdn5IBY6hk6F777E+z7AgZe/dv3kTqN2+3C9+iq2/yxcVyWEsGPB4pYnVpCgEmLWqWkX6iZRoeTq4ZFUVlv556PdzS3IX90RjLxgUZ+PFDE7RN7E2zWg0KksFsMciuVJEnSmZBaVMPBwhr+vGRPm2N3Tu5NnU10qiqpsZEQ4kNlg53t2ZXYXW4SQnxYl14GbjdjE4KIDTSi16g8btXq6RrtTtKLRcetQ4W1PHRxP2obHezIqSQ2wMjI+AB+PlhEoI8es15LaW0jv6aVkhLtR22FA4tBzSOXJqHXqNifX43L7abB5sQnUINPO4GMY4sLl9VayS6v55td+UT7m5g9IpaDhdW8sCqt1Q5yrUrFJ1tyGRBpYWZKJGvTS1m5v5DxCUFcNTyafXlVbM+upJ9Jw4IpCXy+LZft2a2DQr5GDWZ95wdxANQqJZH+BuC3FwJdLjd5lQ2sP1zKntwqBkb5MrZ3EJF+BpTe+EskSVKXkoGcs9T3+wpJifHzrj/8jVXw0z/AFAx9L+7u0bSSFKRkVbaDknoXwcYenOkSkiy2o/30D0i+HFRy1edMsDmc5FU0ctHAcJRKBX4GDVe8sh4QE/+vdubzwYas5vM/3pzDqPgAnrl6EPd8shOlAnoFmZj/3haSwy043aKwoyRJknRm9Q72vAD2h2l9KKpu5LKX1jXf9t76TMYnBPL3mQP494pD/HdlavOxhZuyuXBAGC9dNwQXbpwuNyovmpPpNSriAk0oFfX8Y+YA/u/TXa0K92pVSl66fgifbs1mY0YFExODmD0ihud+SGVHTmWra03pG4xFr+aqYZ7rCB2vpKaRR7/ex3d7C5tv+3BjFpenRPDghf146uiWKZVSQf9IC/vyzQyI9GX+e1uOjl3J+Umh3P3RdjLL6puv8fqaDB6b0Z/JfYNZsr2lg+T8cfEEn0aHqs6yL7+a697cSK31aLHtTeCjU/PJbaMZEOmljUskSeoyPfgTqXSqssrqSCuuZURsQHcP5eQ5bfDT38FWB0Nu6HEBiORAsZ1lfU/PygEYciNUHIGdH3X3SM4ZxTVWrn9rIzq1gosHhPPcj2Ivv49OzYBI31ZBnCabjpRzpLSOlGg/nrpyEIcKawgwavn31YMI8pEZOJIkSd2hT4gPwebWH+p9DRp6Bfvw0eacNuevTS9jbVop2eX1bY6t2FtInc1BSbWVourGLhtzV/Azarn/gkRuGB3LC6vS2nRfsjld/GHxTq4cGk2t1cHyPYXc9uFW7pvWB/MxhZ0j/QzcOTmB8/qFiEzTk7D5SEWrIE6Tr3fmE+SjI9isQ6mA/14zmDqrgyuHRvHk8gPN5102OJLPtuW2CuKA2IH+xDf7SA63sDGjHIAxvQKYMSi82xc+i6obuWPRtpYgzlG1Vge/X7jN635/JEnqejIj5yy0fE8hOrWSgd7SdtztgrXPQVk6jLgFDJ47HnQnf72CGLOCtbkOLk/oWUGmNgLiIW4i/PIvGDwb1N2/ynS2O1RYQ4PdxRtrjrRKzZ6YGMTKfUXt3u/LHXm8dP1Q7A4nKpWZ85NDCbXoMOl6+O+YJEnSWSrcz8DHt47i1g+2caS0DoDz+oawYm9Bu/f5cGMWlw2OZF9+dZtj76/P4qaxcfib7JzM9pqeZGCkLz46NS+saltoGKDO5qTB5sSiV1Pd6KDRLgoHPz87hTWpJYzpHUh8kAmTTk2U/8kVEK6st/HWrxntHv9mVz6PX9Yfg0aF1eEkxKznl9SSVtutJvcN5t5Pdni8v8sNWzLLefCifgyL9Scu0NQmcNcdyups5FY0eDyWW9FAWZ1NFkiWJKkVGcg5Cy3fU0BKtB96jZcURd35ERxZAylzwC+mu0fTrgHBKtbkOHC73SgUPTw9OuU6+Pou2P4BjLy1u0dz1qtqsDf/f3l9y6qlQaOi8phjx6tpdLDzaAr6ef2CifA7/U4ZkiRJ0ulJCDGz+PbRlNXaqLc5CLXoedBDzZwmNY0ODFrPSe41jQ6cbjd2p/f1FvEzajFoTpwJUmt1oNOooFFkkvx8qJjHL+vP1KTQU/qeDpeb6sb23zerGx3kVTTw5HcHuHdqH/qGmXG4XG3OO9Hz3ehw8ftJvU9pfMcqrm6kssGOy+VGr1HidIFBqyLUou/wNjqb48QZ3za7F2SES5J0RsmtVWeZzNI69uRVMSo+sLuHcnIy18KujyFxOoQN6O7RnNDgEBVF9W4OlredMPQ4vtEQPxF+fRYc1t8+XzotAyJ9SQjx4fpRMUzt1zJ53Z1bxZhe7b8Wx/QOZFdOJRa9GoNGxtUlSZJ6ihCznqRwC8NiA4jyN3LpoPB2z53QJ4hdOVUej43pHYhFr2kuhu9tfA2aE273jQkwUnbMtiuDVtVmscvucJFVVscHGzJ5+Ms9fLUjl7yKtlvRxPdTM+0EQaCxCYGs3F+I2w39Iyxo1Uqm9A1pdU5uRQN9Q9vv2Hqi9+WTYbU72ZRRxtWvbeCC59Zw4Qu/8rv3tpJWXMujX+1hyfZcKuttHbpmoEnX3Lr+eDq1slUhaEmSJJCBnLPO0l35GDQqhsb6dfdQfltVHqx7DsIGQ/yk7h7Nb0oKUGJQw09Zjt8+uScYNAtqCmDHh909krOev0HD3y/vz90fi1TupmKZacW1RPgZiA5om05v0Ki4dng0qw4W0S/cgr9J1sWRJEnqqSYmBhPp1/ZvuUmr4nfj4tl0pKzNMX+jhmlJISgV4G/0zr/xoRY9j1ya7PHYZYMj2JBRhuuY5JfrR8YQ6NOyPdjpcrMzp5ILnlvDo1/vY+GmbO5bvItLX1xLWlFNm2tqVCrmjI7B4iHwFWzWMTTGny2ZFUzoE4SPTk2wj47YQCPDY1u25S/alMWC8xLwlDw9tnfg0S5Spy6rvJ45b21qVRcpo7SOez/Zwdyx8fzliz1sOlLeoWsGm3UsOM9zt9a7z0voEdu/JEnqWWQg5yzidrv5Ykcuw2P90al7+LYqp13UcNGZYcBMPL7b9jAalYKBQSpWZrWf8tuj+EZD/ASRleP0kjF7qVqbg8eW7qOy3s7j3+zjr5ckMXdMLEatir99s4//XDOYOaNi0GuUKBVi//6rNwzl2R8O8dSVgzx+OJAkSZJ6jgg/A5/cNprrR8agUytRKRVMTQrh1RuG8Z+VB3nm6sFcMjAcjUqBRqXg4oFhvHPTCIJ8dNhdblxu79taBaBUKpjSN4T3549oznIJMet48KJ+jE0IbFXPJjHUh3lj4tCoWuagRdWN3PrhVqyO1tnMFfV27v1kZ6tsnibR/ka+unMclw4KR61UoFUpuTwlgv9eM5jnfkjlzsm9+evFSfQLs6BVqwg263np+qHcc14CFoOarLJ6DhRW88mtoxkS4weAn1HDH6b14blZKQSdRnZLo93JG2sycLja/jytDhfL9hQwNSmUp787SEnNyWdE6zUq5oyK5flZKcQEiG3WMQFGnp+VwvWjYr2nXIIkSWeMwu320neWblJdXY2vry9VVVVYLJbuHk4rWzPLufq1Dfz14qSe36Zwx4ew51MYdSf4Rnb3aE7ar7kOXtlhY8McH8J9vCAOWpEFS++Cy16CoTd292g6VU96Le7KqeTyl1ta0qqVCqYlh3L1sChCzDoseg21jXa0GhUNNif7C6oorbVx0YAwovyNcoImebWe9FqUpK7WaHdSUW/D7RbZJt/tKWRHTgXxQSYuGhAGiOBHdYOdHVkVONxulu8p5K15w0+64O+p6urXYmmtFZvDhUqpwKBRkl3ewCebs6lssHPZ4AgGRfkS5tt6YaJpbtqeH++fSEKI521QdVYHVQ12FIBWraTO6sANmHVqAjwEYxxOF6W1VlxuMGpV+Bm1VNTbaLA5USoUBPtoUalOb+5WVmtl1hsbSS+u9Xh8UJQvY3oF8vqaDNb+aQpRAR3/mRfXNOJwulGrFK0aKEiSJB3LOzfsSh4t3pJDsFlHckQPn0iXH4bdn0Lv87wqiAMwNFSFWgHfHbHzu4FekObqHwuxY2HNMzD4OlDJl3xXcB63MudwuVmxt5AVR9unvnPTCO5ctI1Gu4uBkb7EBRq5eUJ8u5NXSZIkqWfSa1SEHw1WlNQ0si27HK1KyY7sSl775XCrbUaj4gNICrcQ4WfAcBYE7I/PZBkQqeUfVwzE5XK327673vYbRXwd7dcdNOnUmI5pZf5bdWLUKmWbQJK/UUtnxs/0GiVR/oZ2AzkRfgbK62wEmrSoVKeWbS6DN5IknQwvSCmQTkZVvZ1vduVzXt8QlD15m5LbCetfBJ8Qr6iLczyTRsGgECVL071oq9Kg2VCZBbsXd/dIzloBJi0+Os9BMlEbQUOjXUxW9+RVsSatlGA5UZMkSfJqvgYNkX4Gvtld0KZWDIh6LLtyKrllfPxZXay2vSAOiO1B7U1LLQY1vl5WO8ik03DnZM+1bABmpkSyYm8ht0/qLQMykiR1KRnIOUt8ujUHh8vN5L7B3T2UE0v9HkrTIHmm12aHjI9Us7PYRWaVF3SvAgjoBTFj4JenZa2cLhLpb+CP0/t6PHbTuDi+3pHX/HWQj5aPbhlFhK+c4EmSJHkzrVrFvLFxWPRt5zOhFh39I3y5aGAYvUN8umF0PUOgj5abxsZ5PPbQRUmEemER375hZh69NBn1MQEsrUrJgxf2Y21aCecnh3LFkIgOtyCXJEnqCO/8JC214nC6eHf9Ecb0DsSvJ69sWKth2/sQOVxs+fFSQ0NVGNXw+SEbD4z0kg/jKXNg6d2w7T0YeWt3j+asozlaiDHUouOZ71M5XFJLpJ+Bu89LYFJiMDVWB2MSgggwaYnyMxDmq2/TnlWSJEnyPtH+Rr66axz/XZnK9/sKUSkVXDY4gt+Nj8eoUTE8zr9nz826mFmvYcGUBBJDzby4Ko38qkb6hPjw4EX9GB4XgPo0a9Z0B1+Dhtkjo5mWHEp6UQ0uxO9BVYON85NDCfLRel2mkSRJ3kcWO+6gnljU8eudedz7yU6evGIg8UGm7h5O+za+Cod/hPH3i25VXuzt3TZ2FTvZcINPqxWZHm3tc1CwE+7eDga/7h7NaTtTr8WSGislNY0U11gJtegJNutO2PGitMaKzelCrVQQYvGSQJ8knYae+L4oSWfasYV5/U3abili35Nfi263m5IaKw6XG41KKdtpS5IknSaZkePlXC43L/+czuAo354dxKnMgkPLIfFCrw/iAEyNVfNjloOVmQ4u7qXp7uGcnCE3QvZ6+PlJuPjf3T0ar5BdXs9tH2zlYGFN822Do315Zc6wdluGB8nJqSRJ0jnn+MK8UmsKhVzckCRJ6kzel88otbJsTwGpRbVcOTSqu4dyAm7Y/CYYA0StlrNAnK+SpEAlb++2dfdQTp4pCAbPgc1vQOa63z7/HFdWa+WOhdtaBXEAduVU8cfPdlFZ70U/e0mSJEmSJEmSzhoykOPFbA4Xz3x/iCHRfiSG9uAsl+xNkL8D+l7stQWOPbm4l5ptRU425Tu6eygnL2kGhPaHJTdDTVF3j6ZHK6u1sS+/2uOx9YfLKKuTgRxJkiRJkiRJks48GcjxYu+vzyS3op7rRsZ091Da57SJDJCgRAju192j6VTDQlXEWhQ8t9WK15SaUqpg4h/Fz2XhlVBX2t0j6rGqG0/c4avO6kUBPEmSJEmSJEmSzhoykOOlCqoaeO7HVKYlhRIdYOzu4bRvz+dQXwr9LoGzrEuPQqHgmr4aNhY4WZ3jRR/qjYEw7XGozoM3p0DO5u4eUY/kf4KOE0oFWPReUhtJkiRJkiRJkqSzigzkeCG3282DS/ag16i4dnh0dw+nfVXZsOdTiJsIPiHdPZouMTRURf9AJX9b10ijw0uycgD84+CiZ0BtgLfPh8U3QtYG8JbMojMg0EfL1CTPv7eXp0QQ6CNbi0qSJEmSJEmSdObJQI4Xen99Jr+klnDz+Pie2yHB5RTtrvW+0HtKd4+myygUCm4aqCW/1s1zW63dPZyOMYfBRf+GsfdA3jZ490J4aQSs+x/Ul3f36Lqdn1HLP2cO5LLBETR1mFcrFVw7PIqHLkrCLDNyJEmSJEmSJEnqBgq31xT36Bmqq6vx9fWlqqoKi8Vyxr//5iPlXP/mRqYlhzJvTNwZ//4nbceHsPtTGHU7+PXgGj6d5Jt0Ox8fsPPWhQamxnrhB3y3Cwp2Q/oPkLUeVBoYcTOMv190G+uBztRrsc5qp7TWRp3VgUmnJthHh7GnBlAlqRt09/uiJEmCfC1KkiSdO+SnES9ysLCaWz7YQt8wM3NG9eDgSPZG2LUY+lxwTgRxAC7prSatwsWCHxtYeImCYWFe9tJSKCEiRfxrqIQD38CWt2HbBzDlLzDilrOq41hHmHQaTDovDM5JkiRJkiRJknRWklurvMT27Aque2MjAUYtf5iWiFrZQ390hXvgl6chrD/0mtjdozljlAoFdw3REmdRcsOyelZknLjjUY9m8IOhN8IVb0DsGFjxILw+ATLXdffIJEmSJEmSJEmSznk9NBogNXG63Lz1awazXt9AiEXPXy9O7rl1cY6sgR8eEYV0B14rsjzOITq1gj+P0jEoWMXvf2jgzh/qSS13dvewTp3BD8YsgEufE1uv3rtYFEUuSe3ukUmSJEmSJEmSJJ2zemhEQKqzOli+p4DX12RwuLiWCweEcd3IGDSqHhgcqS2G7R9Axs8QPhgGXCVqrJyDdGoF9w7TsjbXyeJDdi74rI7kQCUTotQMClbRx19JjEWJXu1FrdgDE0RR5IzVsGMhvDxStJMfcTPETwKlqrtHKEmSJEmSJEmSdM6QxY47qKqqCj8/P3Jyck67kJzb7abe5qS83k5xjY28qkYySuvZk1fDrrxqbE43AyPMzBwUQlygsZMeQSewN6KoLUBVfhhVwTZUhTtBrccePw1nWAp4UYyiKzlcbnaUwLZiN2kVUH5MU6tAPYQZIcykINgAQQYFgXrw14O/ToGfDnx14KsFraoHPaFOO+qsX9CkfYeyOgeXIRBH3CSckSNwBSXj8o/DrfcHxamP2Ww2oziJ+3fma1GSpLbka1GSegb5WpSknuFkX4uSdCbIQE4H5ebmEh0d/ZvnKTQ6wuc9jybwt89tj6uxFmddxSnf/3RYdBBhPvnsH7tTBKak9tUqTFQo/Tv9uvHOLJ6v/ytazmxdnqTgjmXiTHm/jtWZv73V7GS7bZzsa1GSpFMjX4uS1DPI16Ik9QyyI5zUk8hATge5XC7y8/Nxu93ExMR4xapHdXU10dHRXjFWkOPtaj19vCe72tH0Wjz+/J7++Lraufz45WPv3Md+uq/F7nIu/x50Jvk8do7OeB67+7Uofxc6Rj5fHeNNz1dPeZ+TJJA1cjpMqVQSFRVFdXU1ABaLpcf/0WniTWMFOd6u5m3jPV7Ta7E93v74Tte5/PjlYz+zj/23Xovd5Vz+PehM8nnsHGfieezq16L8XegY+Xx1jHy+JKljemDlXEmSJEmSJEmSJEmSJMkTGciRJEmSJEmSJEmSJEnyEjKQc4p0Oh2PPfYYOp2uu4fym7xprCDH29W8bbwddbY/vt9yLj9++djPzcd+PPlcdA75PHaOs+F5PBsew5kkn6+Okc+XJJ0aWexYkiRJkiRJkiRJkiTJS8iMHEmSJEmSJEmSJEmSJC8hAzmSJEmSJEmSJEmSJEleQgZyJEmSJEmSJEmSJEmSvIQM5EiSJEmSJEmSJEmSJHkJGciRJEmSJEmSJEmSJEnyEjKQI0mSJEmSJEmSJEmS5CW8PpDz8ssvExcXh16vZ9SoUWzevPmE51dWVnLXXXcRHh6OTqcjMTGR5cuXn6HRSpIkSZIkSZIkSZIknTqvDuQsXryY+++/n8cee4zt27czePBgpk+fTnFxscfzbTYb559/PpmZmXz++eccOnSIN998k8jIyJP+nm63m+rqatxud2c9DEmSToF8LUpSzyBfi5LUM8jXoiRJ0rlD4fbiv/ajRo1ixIgRvPTSSwC4XC6io6O5++67efDBB9uc/9prr/HMM89w8OBBNBrNKX3P6upqfH19qaqqwmKxnNb4JUk6dfK1KEk9g3wtSlLPIF+LkiRJ5w6vzcix2Wxs27aNadOmNd+mVCqZNm0aGzZs8HifpUuXMmbMGO666y5CQ0MZMGAATz75JE6n80wNW5IkSZIkSZIkSZIk6ZSpu3sAp6q0tBSn00loaGir20NDQzl48KDH+2RkZPDTTz8xZ84cli9fTnp6OnfeeSd2u53HHnvM432sVitWq7X56+rq6s57EJIknTT5WpSknkG+FiWpZ5CvRUmSpHOX12bknAqXy0VISAhvvPEGw4YNY9asWfz1r3/ltddea/c+Tz31FL6+vs3/oqOjz+CIJUlqIl+LktQzyNeiJPUM8rUoSZJ07vLaQE5QUBAqlYqioqJWtxcVFREWFubxPuHh4SQmJqJSqZpvS0pKorCwEJvN5vE+Dz30EFVVVc3/cnJyOu9BSJJ00uRrUZJ6BvlalKSeQb4WJUmSzl1eu7VKq9UybNgwVq1axcyZMwGRcbNq1SoWLFjg8T7jxo3jo48+wuVyoVSKGFZqairh4eFotVqP99HpdOh0ui55DJIknTz5WpSknkG+FiWpZ5CvRUmSpHOX1wZyAO6//37mzZvH8OHDGTlyJM8//zx1dXXMnz8fgLlz5xIZGclTTz0FwB133MFLL73Evffey913301aWhpPPvkk99xzT3c+DOlcVFMAZRlQtBf8YiG0P1giQem1SXKSJEmSdG6qLYaKLMjfDpYICB8M5ghQefU0+/Q5HVCTD4V7oCoXwlPAPw7Mob91T0mSJOk3ePU7zKxZsygpKeHRRx+lsLCQlJQUVqxY0VwAOTs7uznzBiA6Oprvv/+eP/zhDwwaNIjIyEjuvfde/vznP3fXQ5DORRVZsOgqKE1ruc3gD3O/hrBBoFB039gkSZIkSTp5VXnwyfVQsLPlNq0J5iyBqBHnbjDH5RSBrYVXgrWm5fbQAXD9YvCN6r6xSZIknQUUbrfb3d2D8CbV1dX4+vpSVVWFxWLp7uFI3qahCj6fD4dXtT1mCobbfgHfyDM/Li8kX4uS1DPI16J0zrLVw7L/g10ftT2mNcGdG8Ev5owNp0e9Fitz4NWxYPXQSav/lXDZi6DzOfPjkiRJOkvIfRySdCbVl3oO4gDUlUCVLFQoSZIkSV6hrgT2fub5mK0OCvee2fH0JKWpnoM4AAe+hrrSMzseSZKks8w5mu8pSd3E3nDi4/VlZ2YcknS2cDpEcNTthj4XyDpTkiSdOU4bOO3tH68pOHNj6WlqCts/5nKC03rmxiJJknQWkoEcSTqT9L6gM7feL36sgN5ndjyS5M2cdlh8I6R+J77uPRWu//TcrUkhSdKZpTWBObz9gE1EyhkdTo8SNqD9Y6Yg0MptVZIkSadDLl1K0plkDoNJD3o+lngR+ISc2fFIkjfb/AakfQ/nPQpTH4OM1fDL0909KkmSzhXmcDj/Cc/HIkeAb/SZHU9PYomA2HGej019TDx3kiRJ0imTgRxJOpNUGhh8HVzyrFiRAtAYYfRdMON5MAZ06/AkyWtYa2D105B4IUSPFN1hBl4N6/8H1fndPTpJks4FCgX0OR+ufEsELgBUWhg6D659/9xenDEFw1VvwfCbQa0Tt5nDYOar0O9SuQ1WkiTpNMn8c0k600yBMGy++ABqrwe1HkwhoNF198gkyXvs/UIEcwZe03Jb/6vg0Hew/iW48MnuG5skSecOg78IIseNEwWO1ToRxNAYuntk3c8SAdOfhHH3iZo4TVvRFIruHpkkSZLXk4EcSeoOSqVsMy5Jx0kvruVwSS0T+wRj0KpOfPL2DyBymPjA1ERrhIRpsHMhTH1EfpCSJOnMUChaMnKk1jR68D9zLdglSZLOFTKvUZIkSep2H23KZtqzv3D7h9u46IU1lNfZ2j+5Oh/ytkKvSW2PJV4IjVVw4JuuG6wkSZIkSZIkdSMZyJGk7tBQCaVpkL0RivZDbUl3j0iSus3+/Goe+Wov05JCeOrKgVTU23ngs13t3yFtJSiUEDm87TFLBAQnia1XkiRJx6svh5JUyN4AxQegrrS7R9Sz1RRD0T4xXyk7LALlkiRJUreTW6sk6UyrLoDlf4SDx2QMhA6AWQshIL77xiVJ3eTZHw4RbNExb2wcaqWSeWNi+d9P6WzLKmdYrIcC4KnfQ0gS6MyeLxg3Hra/Lz5w6H27dvCSJHmPqjxYugAO/9RyW+RwuOZd8JPbf9ooTYfF10PJIfG1QgEDroEL/i4KF0uSJEndRmbkSFJns9eLyWJVrih8eCxrHfz0j9ZBHICivfDRNVBTeObGKUk9QHpxLT8eKOaKlEjUR7uYjOoVSLS/gTfWZLS9g8sFWesgPKX9i8aOBacN0ld1zaAlSfI+jVWw/IHWQRwQ2zQ/nXf6mbE1ReJ9v7bo9K7TU1Tnw4czW4I4AG437PkU1r0AjsbfvoatTjwnVXlgb+iyoUqSJJ2LZCBHkjpT+RH49gF4cQi8MBi+XiBSkd1ucbyuGHZ/4vm+pWlQU3DmxipJPcCS7bn46NSM7hXYfJtSoWBKvxBWHShuWyun5ID4QBbav/2LmoLBPx7SfuiiUUuS5HXqSiD1O8/H8rdD/Slusaorhd2fwjsXwHP94d2LYN+XYguXN6vIhKocz8e2viMCV+1xuUQ2z9d3ibnQi0NEJnJFZleMVJIk6ZwkAzmS1Fkqs+Gd6bBrETis4HLAvi/gralQmSXOsdWK29tTlXdmxipJPYDL5ebL7XmM7hWIVt367Whc7yDcwLe781vfKWs9KNUQ1PfEF48cJmrpuFydO2hJkryTtaZlUcWT+rKOX9NWD5vfhC9ubQlSlB2Gz26CnYvAbj2VkfYMJwq6OBrBcYIMm8oseOs8EdByOcScaMeH8M6FUNlOcEiSJEnqEBnIkaTO4HLBvq88p1Q3VMDW98BpFzU9VNr2r+MX1VUjlKQeZ29+FYXVjYzp1bYOjsWgITncwo/7j3tN5WyCwARQ60588cihYoW9aG8njliSJK+l9xVF0ttjDOr4NeuKYe1/PR/7+Z/evc0qsHf7x7Qm0Bg9H3PYYPMbnosi1xTAoeUnDqhJkiRJJ0UGciSpM1ir29a9OVbqctGpyhQKQ+d5PidsEJgjumR4ktQTrTpQjEmrIjHMc9HiITF+bMgoo856TBZb3nYI7PPbFw9OEsGeI7900mglSfJqpmDof4XnY7HjxPGOqi0WizSe2BtOfbtWT+AX034wZ9Qd4NNOsePGCkhd0f51938lspMlSZKk0yIDOZLUGVQa0FnaP66zgEoNWgNMfACG3AhKVcvxmLEwexH4hHT9WCWph/jpYDGDovyaixwfb2iMP3anm3XpRz8MNVZB+WEISvjti6s0EJIMGTKQI0kSIiP2gn9C/ytF96UmvafClW+AKbD9+7ZHrf+N47+ROdiTmcNhzhKxTbWJUg0jboNRt4O6nexi5W/Mh/R+4jqSJEnSaZF/SSWpM2hNMOYuSP/R8/ExC8DgL/7fHAYXPg0T7hdZOlofMAWB0UObZUk6S1U12NmXX8UtE3q1e06oRU+IWcfGjHIu6B8GBbvFgZPJyAGR5bb3c3A6RCBVkqRzmyUcZrwA5z0sAsM6s8jEMfid2vVMQWCJEB2ejhfQC4ynEBzqSQLi4frPRGaRrU7MY3xCxJynPcYAGLsAltzi+fjoO0Bj6JrxSpIknUPkzFaSOkvYIBh9p2hlWl8OZeni9uTLIWZM63N1PuKfJJ2jthwpx+WG5PATrNwCSeEW1h8+mpGTv0OsgFsiT+6bhPaH7e+LOjkRKac3YEmSzg56i/jXGczhMGshvD9DBDpAZOFEjYJLnxULN97OFNjxbKX4idD3Eji0rPXtw38nMiVPRlOh+nYyNiVJks51MpAjSZ3FaYc+54sVPp8QEdhxuaA6D1JXQu/J4Bfd3aOUpB5hY0YZwT46Qswn3nqQFG7hl9QSKups+BfvB/+41tsSTySwjygunr1RBnIkSep8CgWEp8AdG+DwT6KgsjkcstbCweViTmCJOPWMH2/lEyoyn8bfB3u/EBmRA64WdXd+K/u4pgiK98P2D0ChgmHzILhv5209r8qFwr1QsEtcN3KYWByQASNJkryMDORIUmeozIYPZ4q2o03Uepj5Gmx+DXI2i0nI/O9Exx1JOsdtPlJO3zAzimNrVXiQdLQQ8o6cCs4r2is+CJwslQaC+kD2ehj9+9MZriRJkmdKlfhbYwqG/V/Dnk9bjv34KIy/H8befe5tn/YJFv+iR578fWoK4cvfQ8bPLbft/Qz6zYBL/ysCRKejNA3ev1R8nyZ6X5j3jVh8+433I0mSpJ5Ehp8l6XQ1VsN3D7YO4gA4GmHpXTD2HvF1bTGseEicL0nnsEa7k/0F1fQJ/e3thcFmHX4GDdszy6HkkMjI6YjgJBFIlSRJ6goup8i+qS1qHcRpsvZZ8bdL+m0Zv7QO4jQ5+A3k7Ti9a9eVwpKbWwdxQNRK+ni2aI0uSZLkRWRGjiSdrvoySP3O8zFbHTRUiIKIdaWQ/gM0lJ/6/vym6+37GupKRNp2YAKYT3OVSpLOoL15VThcbvqEeG47fiyFQkFCiA/bDxeA0wZ+cR37ZkGJouBxdb7Y4iBJktRRLpfIvM1eD0X7IXwQxIwWLbhtdeLfwaXt33/jK2J7p7cW+a0uEB0DU1eK+Ua/GWK7WGfVGgJRW3Dza+0f3/gKxI0/9fqC9aViO5Un1fkiECffIyRJ8iIykCNJp8thBber/eP15aIzVV0puN1i9a4jXE6ozIGD30L2BrG1pPd5kPETrHsOIofDrA/lBETyGjuyK9GplcQEGE/q/IQQH5buKMepUqDyj+3YNwtKFP/N3QrJl3VwpJIknfPcbijYCR9cBtYaUcA4egQcXAZZ68R2n5Q5sP299q9RVyLq6HljIKcqFz6/GfK2wLTHxTbxnx4Xc5OUORA2sHOKOrtdYG9o/7i9HlyOU7++vfHEx621p35tSZKkbiADOZJ0uvQWMZGrLfJ8PKhPS2vS0P4iqNMRRXvh3YvBdswkY/MbcPnLYK0W20a2vA2THxT79CWph9uZW0l8kAmV8uTqEfQO9qHeoSDDkEgfvW/HvpkpSPzL3SIDOZIkdVxNASyeI4I4aj1c/gp8e5/I0GnSUAFxE6A8w/M1Eqd3/L2/J3A6YfuHkLMRLnwaMteKRaUm+76E2PFw9VuiyPPp0PtD8hVQfMDz8UHXino2p8oYABqjCAgdT6GQi2GSJHkdWSNHkk6XORwu+IfnY70mi73xTpsoiDjxz+DuQEZOTbHY0207bqXI5YDv/iTanQNseVOs+EmSF9iTW0VckOmkz48/eu5ebcqpfcPAPqJ1uSRJUkfVFLYsxgy4CnYuah3EATjwDfSfKQIFxzMGQP8rvbMrUk0+bD0apNGZWwdxmmSthdQf2t7uckFVHpSkQkUW2E6QbQOgUkHKdZ6ze/xioO/Fp1eM2CcUJv7R87Gh80WxakmSJC/ihe8qktQzOF1u6m0OnC43JF4IsxaCf7w4qDPDmLtFoGXjq9B7Ksz7FhorwOZhNag9DWWiy4InjVUi5VulFf/f0S1bktQNqhrsZJfX06sDgRyTTk2Yqpo9nGLHt8AEURvB7T61+0uSdO46drtPn/PhwDG1cHyjod+lED8BVj8N85bC0HminbVCKYIPv/sBOroltKdwO6GhUmQU7f+69bHwwZB0GUSNgM2vi+3jTerLYccH8MZEeHkEvDQc94o/twTE2uMXAzf/CKN+D8ZAEVwZdx/ctBx8o07vsah1MHQuzHixJXvIGADTnoApD3VuvR9JkqQzQG6tkqSTZW+Aqlzce7+AsnScg29EYwrAemQDOpUCRfwElDd9K9KvyzMg81cRhJn6CBTvh0+uh8kPiSDPyXLaT3zc0QhKtaiT441p29I5Z19eFdCSZXNy3MSRzx77Kaa+ByaIbYjlGRDY+9SuIUnSucknRGTUupyAQrwvG/zFViNHo9huFNALBl4DdcXi703vKbhDklGYQsDo392P4NRpfSB2nNi23RTQChsI5z0MBbuh5CDETxJFiJtqBbqcYsvVsvtbruO0odj+Pu6yNNxXv4/SHCLqCzZtV2usEtcqPgDBfWHMAphwv8jqMQV13rZxUxAMvRH6TBPfX6UVGUBKVedcX5Ik6QySgRxJake9zUFFnQ2XGwJ0bkw5q2HxDSjcLpj8ENrUb2DzGxw7vXCPuw/FoGtF0MaTdc+Lbg8nyxgg/tWXtz2mVIsVK6cVpv/TuyeL0jljb34VOrWSCN8OFP2sKyXencvXdYm43G6UHU2vDzyayVOwSwZyJEnqGHMEjLxNZNcW7YXYsSJL5IdHRfBBZ4Zr3oMlv2uVQavQmXHfsASFfrj3BgpMQTDtMVj2ACTNgKocmPJXWHKrCI432fAi3PClmJPUFMDPnrebK7LW46jKR2mtgc2vQvFBscC15ObW7b99QmDuNxDSr/Mfk6yHI0nSWeKs2Fr18ssvExcXh16vZ9SoUWzevLndc9977z0UCkWrf3q9/gyOVuqxakvEpKJoHw1lOfxvVRoTn1nNjJfW4qjMhc/nixUnnxAIiBcFh4+jWPc8lB8Re7E9qc4Hx9GtVS6n2D9euBdKU8U+/Ios8XVltlgtMofDhf/2fK1Rt0PedvjdSghJ6pznQJK62IGCGmICjChPstAxANV5xCqKqHcqyak+he1Rel8whYjOM5IkSR2hM8HYe2D6U5C6QnRuSl0hgjix4+D6z2DTG223QVtrUHw067e3E/V0gX3gwqcgPAUmPAArH24J4vQ+D65+B654A2w14rHa6tsuPgX3hUuehWs/RO2yig5Yh3+GvhfCd39sHcQBqC2GxddDTTtNJI5nq4fyTCjcI/7rqaBxE6ddzLEK90LZYZENJEmS5IW8PiNn8eLF3H///bz22muMGjWK559/nunTp3Po0CFCQkI83sdisXDo0KHmrxWnUzxN8n4up3jzX3o3aI2g1GCoLeK+qf/g8v7DiNXXYcjbIAIrIPaE717c/vW2vCVWkmoLYPkfRZCmiVonUnkbKsVE8Pu/gPboJNE/HjR60SJz02siGDTqdrE3/ablsOpxsUXLLxYm/UlMqnRmkbEjSV5iX37VSbcdb1adT4yyFJxwoMxJrO8prEH4x4mJuyRJUkdZImDEbbj6XYLC3oCi1xTRetvlBFsdpK/0fL+GCihLA58wUHtpV0m9hcbw4Sirc9GYAlBMexx8I0FtgIzV8O390Fgpzo0bLwI2138qFr4KdkHxPtGNatUTUHEEBYitaNMeB50PFO3z/H3LDuOsLaYCXw4V1VJRZyMxzEywjw5/k7blvJpC+Pkp2LVIBGlUGki5QXTyPL5wcl2p6ML1639EEwmFAhLOh0v+K+rzSJIkeRGvD+Q8++yz3HrrrcyfPx+A1157jWXLlvHOO+/w4IMPeryPQqEgLMxDVXzp3FSZDfu/EsXuMteJDlMjbkFfkUov/1h0m98QbcOb6H1F9k576kpg72eQ+j1c8y58eQcU7hbHBl0nMgOOrIEvbxetyac/BSv+LFaGQExCRtwCMWMge6MI5MSNg+sWg6NBBIJMQV32dEhSV7E5XGSU1DEuoYO/vzUF+Bm1+DbCgXInF/Y6hQ9E/vGQ8XPH7ydJ0rnBVicWbHQWUB03PXY5oa4Ixa//RbHjw5Z6MGGDRDtyn7C2WSVNyo9AQB/wj+7a8XchTVUWqiW/a8lqVChFO/De57XeYpW5FhZdDVMfhSW3QMxouOQ5+OByUT+oSXmG2E4197gCysdR1Bbit38pvaKnc8hl4vMN+USHBjJjcCR+Ri00VsP3f4W9n7fcyWmHbe+KLJ1L/tNSxNjphN2fwqq/tZzrdkPaSvhoFtz4FZjbyaaWJEnqgbx6a5XNZmPbtm1Mmzat+TalUsm0adPYsGFDu/erra0lNjaW6OhoLr/8cvbta2c1QDr7uVwiHdpWCx9fBxteElumPpsHGavRqtUi46WpxgaIrJjoke1fM2qEKNhXuBs+mQMznj96+yiRSWOtgR8eEbed9yh8eVtLEAfEJKRpL372Rkj7AaoLRA0cS4QM4kheK724FofLTWxARwodA9X5KEwBxJiVHChzndo3D4gXHyROFISVJOncU18uAhCf3QQLrxTdp8qPiPmByyX+v+QQbHgJxfb3W4I4IN7nP71RZJe0J3ywCPI4HV3+UDqdrQ7KMlB9fE3rraluF+z6RMxRBlzZ+j6V2aIItE+oOL7kZpjkoe23Sgt6v/YLGSuUKN0u1L/+m/CPpjIp/y3+MkrHnPpFWH7+C+7DP4mFs8M/er7/3k/F8Sa1BbCmna3qxfuhMqu9Z0GSJKlH8upATmlpKU6nk9DQ1hH00NBQCgsLPd6nb9++vPPOO3z99dcsXLgQl8vF2LFjyc3N9Xi+1Wqlurq61T/pLOJoBJcDNr/Z9lj6jyjSV4kVN40RBs8Wt6ethH6XeO4SpTVB8mWQ+p34uiJTBG7mfA6DrhErV06r2FvvFwO1RZ4LGQNsfEXUvlk8R2z7Kj/SKQ/ZW8nXovc7VCR+ZtEBHSh0DFCdB4ZAYixKDpQ5T+2b+8eL/xbtObX7S83ka1E6azRUwvoX4b1LxHt7/g749Rl4fZL4YJ/2g2g3XnoItr3n+RoVR0QXK08dKXtNEcGKhjKw13X68Lv0teh0QGUOVOe2Xmw61s6PxHbz4xUfaNmqVLxf1PtTaVufk3I9HPlFtGv3ZOg8kdl8lGLLW1ByAOW2d1Du+gjFgW+h/DDc8BVc+hyEDmh9f7dbbG1rYqtr/XWbMe9v/5gkSVIP5NWBnFMxZswY5s6dS0pKCpMmTeKLL74gODiY119/3eP5Tz31FL6+vs3/oqO9NzVW8kChap2SC6Ib1Ni7Yc5n4BMsMmCK9oh91Bf8AyxR8PPTIiU4ZkzL/WJGi6J/q/5+tE3pURWZYvXKN1LsBXdYxb5tS4SYhLSntqglJTj9B1HYuLqd1O1zgHwter9DhbUE++gwajuwq9ftEjUQjIFEW5Tk1Lipt59CwWNLuGhzWyQn66dLvhals0ZtEax9tu3tKbNh3QugcIvaLihOXEC3IhMu+GdLTRaNEUbeCsPnQ2Um7pD+4u9PJ+vS12JtoajrV3iC4Lej0fPtvlFQX9bydXW+CHbpfaHPBTDzNRhwlehyNXy+6FxlONp5U+8L4+6FIXNEcO1Y+74Ubd6v/UD87D6eDW9OFj+rUb+HITe2Pl93zIKbWi8eT3tkjRxJkryMV9fICQoKQqVSUVTUuqp9UVHRSdfA0Wg0DBkyhPT0dI/HH3roIe6///7mr6urq+Wk9azialmhMYdDwjSRebP5DVh0TctpAb3g4mdg75cw5S+i0HBtsahdc+HTUJUr0o6X3i1uP5bWR6QWW2sgKFHUuhn1e9j6Dgy4uv2h+YSI/d9N9n8pJkcuJ/hFddYT4DXka9H7pRZWE+XfwWyc+jKx3dAYSJRWFKZPq3AxOKSD7XwVSjFRLznQsftJbcjXonTWyFjd9jaFQswFvr5LbItyOUCBCM60F8wxBsCOD2HKwyIQERAPWetExo/TIRaNThREOEVd+lp02EQ2jiWy/XPUesSTcwytSTRlKM9ouc0SAVMfE4Ebgy98cZvYggXi+T7vEbjhC9He3GGFPZ+K62iOe79orISkBbB0QevrV2SK2654XTzv5RkQOQyMwS3n+ISIIshb3277OIyBIpC0f6n4mfuEiuYTkiRJPZhXB3K0Wi3Dhg1j1apVzJw5EwCXy8WqVatYsGDBSV3D6XSyZ88eLr74Yo/HdTodOl3nv/lKPYTGAMmXQ+KFYtJQUwT7vxb/jlWeISZ1l74A9gZwO8Ub/ZZ3RcqxMRB2LoL+V0BQX2goh71LxOTNaRVBHBAdrBZeCbMXQVm6WL0z+HtO9x1xK+z6qOVrlxO2vydWqOZ+A6HJXfa09ETytej9DhbVMDTGv2N3qj0aqDcGEKVXogAOlTs7HsgB8I2WGTmdQL4WpbOG20PNLWPQ0SCDQgRxhtwIPhEwdK7oKHk8v1jxPp+xGqry4Mo3RECiaL/IxO13OQp319TH6drXohustSIAFdCrdeCkScr1YutZE4M/XP2uyDaOHCoyiQN7i0WsT+fCRc9A7lbxXKavguwNYgvUqifEPKg0tWU71bCb4NfjsqXiJogt6uUZ4nkfcJUInBXvE/O2dc/DsPli7nT1O2AKbLmvxgAT/yh+tuk/tNzuEwqXvSgaUJQcElvArn5bZGEfH0iSJEnqQbw6kANw//33M2/ePIYPH87IkSN5/vnnqaura+5iNXfuXCIjI3nqqacAeOKJJxg9ejQJCQlUVlbyzDPPkJWVxS233NKdD0PqTr0mw/I/Qdr3cM17oi2lJ8N/JwI4W94QW5wihsKNX8CeJdB7kphQbH5T7KM3h4kVppD+8PGs1tepOCKydixRYtJz/adiAtE0SVJpxN5wvS9krW+5X+J0WPMf0T7z41lw80qRRSRJXqDW6iC/spHLB3ew9XjN0XpnBn90KgVhJgWpFadY8NgvVnxwcLlAec7tLJYk6Xi9Jre9zWkT2Te1RRA+SGwfenc6XPGa6NS05/OWAFBof7jqbZERcstPohbOD4+KLN2IwRB5OajVuOvLUPjHnslHdvo0BpHFUpkLF/8HfvxbSwdOhVIsXA2aJRaq+pwPplCxILbhZRHISZoBkx4S29M3vyW2pudsbNnO3m8GjF0gMpnry2H7B+L7pH4vaguVprfeuuUTIgI5+78W19KZxQJabQlEDYfZH8OaZ8TWrUGzPHegsoTDla+LOVhZhsi6qSuBlX8VjS9A/Pw/nQd3bRadRSVJknoorw/kzJo1i5KSEh599FEKCwtJSUlhxYoVzQWQs7OzUR4zYa+oqODWW2+lsLAQf39/hg0bxvr160lOPreyGyRwu92U1FgJKD2MOu3oCpDODH7RIsvGdcwK2rCbxGrRJ9e33FZxBA58DTd+KVaUPrhMbAFpOpa9QWyhSrywbSpvyUHY/j6s+ReEDYZx94lsAY1BdNDa9wV896eW86OGi2M6M5iCxYpSbbEM5EheI61IZKV1eGtVbaGoFXW0s0mkj5JD5ada8DhWbI2ozBJbHyRJOreZw2DUHbDp1ZbbGqtEC3KtSQRsfvyb+O9Xd4gFnes+BnujeP81BYntU3nbxTnHLgRVHIED38Dsj1FYIsQ2UWPg8SPouUwh4rn49l7x93fcvWJrucspMo1cblFA2GkF3xgxZ2kK9ID4/5D+MOtD6DtdHD+2aHLJIREoufQF0fmrsUps1Rp7j1jM+uERETBSKCHpclFLZ/2L4r8Hv4UdC1uu1fRcX/M+LoUKh7Ueu6IGk4+HAtTGwJZ/b18g7ns8t0sEjCY+0HnPpyRJUifz+kAOwIIFC9rdSrV69epWXz/33HM899xzZ2BUUk9WWW/jx/1FFFVWc2fJ22KVbeC1IkAy8BrRqSprPax/Qdyh3yXw0bVtL+RywLL/E4GYpiDOsTa/DvO/h23viGBPE/94GHGzaGteWyxWkapy4HcrRdaAMVhMcNR6SLkRYkaJlOOkGSLgo/MR+9clyUukF9cCEOHXwUBOTSEYApq/jDIr2FhwioEc36O1I0pTZSBHkiSRFTvxAZFRsvZ5qC8R2SDhg+DaRVC4t2VRx+UUWbdb3oboUTD4OkjNhOB+0PdikW0y5Eaxrbqplo7LCcsfgKveEpke3kSlFnX6GivF1ysfFnX6QvuLLVc5G2H+CkjbKLZeJV8GKddB/k7Y/5WodVO8TxRLrszy3PmqNE38PY4YKs7xixZbyD+eBdcuFFuhFEqoKxYZ0QG9RPDt2CBOE0cjrPk3yhG3ot3yFqoLn8aVU4SiOh9FxGDwi2udpeO0n7jleEVm29tcTlCewrZeSZKkLnBWBHIk6WQ4XW6KqhtpsDvYkVWJ0+3mqiGRKFLHg61GBGqO3S8/aJYoZLz+RVHP5thAzLFKUz23HQVxn6I9Yl/48qMrO1MeFqt4OVvEhNA3SnyfhgqoKRBtOwfPhiE3QOoKUQvn8/mt96cbA0UmUEOl6AaRs0mMIXK4mKjIfd1SD5NeXEuIWYde08FJcHVhSzcTIMKspDDdQY3NjVmrOMEdPTAFg9ogVoITp3fsvpIknZ1MQZAwFaJGiGCLzgJqrdiCWXtcp8j+V4pgxZFfRTDYL0a8N39zD9SXigDPrIXw639F0V2AymzcTgcKbTvzhJ7MJgLwGAPhwqdEjcCczRDYR2ToOJ2iu1T6Ktj7hciqiRsH130iOnjmbxdBmL1ftP89Di2H+ImiYHRtsbgWQFV28wJa3fRn0SdOQdXvMvH925O3DSY/COPvQ7Xwipb6hCDak1+/WMy5ADQG3BFDUeRt9fzQ4yZTXWslSG0Vi3w7PoTqPOh7idjidQ42nZAkqWeRgRzpnFBvc7AurZQV+wq5YVQMyRG+pBXX8OiydF6dmILqvYva3mn3YjGx840RH/5ORHGCD5ROu8i86XOBKPgXNhDentYSGKopgMVzYOKfRJ0d32jQmUTqdnBf+PkfbYsM1peJbV6XPAcfHdNdS6kWbT37XSS6ZUlSD5FWXEtkR7NxQGytihza/GWUj9gqe7jSRUqHO1cpxCS+9FDHxyFJ0tlNb2n9tdshgr8KhXi/7ncpxIyGxTeK+jA+YWIr1e7FLffZ96UITFzznnjfrjkaCNL5tG6F7S2MgaD3E92gVjwouk8Nng0ak2jqYA4TW89SV7TcZ/encHCZaBG+5Bbx3J2oJplSLTqBBsSDrUHMuZJniK1r136IzeFkgy2RiXYbqsby3x6zKURsdT82iANQtBe+fxgufwl0PhQ7DGgnPIbfJzPaXsMSwX5VP37+ZR/3BG1H9d3/tRw78I14Hm5aLjM7JUnqVrLao3ROyCmvR6dRoVOrWLq7gDd/zUCvUaFXuVHu9JCi22THhzDjeYgd2346bXiKKNTniVIlOjasex7G/0HU2ll2v+fsnrX/Ff/95DooPgg1+eJDZ+5WGH6z2Jd/7QdipWv8H0Q2js4E17wPvc8T93U54MtbRecMSepB0opqOr6tymkVmWrHZuT4KFAAaRWnur0qSmTkSJJ0brPWimBL/k5RWLehqtXhaqUvrpI0UUMHYNg8+OVfouYLiIK+xwZxmjisIiNnuGi6QVAi2BtxN9a0PbenMwaK4sB7PoVRt4u50MpH4LN58NOTYlHp2CBOE1sdbHxFdLXSW6D/Ve1/j6FzRSbN9w8BbrjmXZGZ/MZE+Gwuqt0fMSjAhdLgB7b6ExcgjpsgMqsbqzwfP7gU6kqobrDz8Ff7eGaXltLLPwL/OHFcocDRaxpZMz5jwbJiruijRrXCQ52c6nwRwLLWtj8WSZKkLiYDOdLZy2GFhkqsjQ3UNDp44LNdfLQ5m5HxAaxLL6G0xkpisA5FdX7716grEanELidMfaztca0JLnoaokd7zoCZ/BfY9YlIF85aJ9pmVrcTZHE5oeSACPzkb4N3povJ5ZVvgrUKFt8g2nd+PBsKdonJTmWuWCXrNRnG3CWu43aLFTFJ6iEa7U5yKxo6npFTWyL+e0wgR6dWEGJUkFZ+ip2rmgI57W2VlCTp7FdTKIrvvjQc3pgELw2DL24W3aYQiz83LjyEI+ECCB8MV78n6rmc/4ToSlW8XyyytCd3q6ido9bD1EdRFOzA2Vh9Zh5bJ3K6XLiNQaDUim3kq58WwRuAgDiRedOewz9B/GSRcWMOE3Vwjhc1QnSj2vq26MipMcCHV4j7ut3gdqNKX0nIp5eiri+Gr26DfV+Jxazj6f3ggr+LZhLtcTnBaaW01srK/UUs2lHGTb9aKLr6aw5d/RMHrvmFRTF/I7vGxZIrzEQrSuC6xSLYdLyD37Q8F5IkSd1Abq2Szj62WijPhI2vQulBtGGDSR5yMxcnB/DepgJww7h4XyZFQlmgGfTjIONnz9eKHA47PoCEaWKP9KyFYq93TQFEpEDC+bD63yKYM/drOPCtKABoDoehN4q07IgUKDoABTuh13knHrvGKFb/3pwiHoclHLa8JdqdHuvwT2LF6+JnxFh+eBRmvipWlSoy227FkqRudKS0DjcQ2eGOVUXiv8cEckBk5aRXnkbBY2u1CNL6hJzaNSRJ8l62OlG/Zeei1ren/QBLbsF17Yd8sqWcOYNMqL65Cwx+4v1cpRVZtsUHRMHfo530PFIoxBaf6z6BX56GSQ/htjd06cPqbK6aYpSfzUUx5S+QdIlYSGp1gkM8J+1RqkWnwLUvQNp3MPN1sVV2z+fi+Rl8nXiOUr+DK98Qdf52LxZ/n4/XWAm7PobeU0XHz7H3iGzkvUvE3/Jek8W2t2/uFQWS22MMxKEy4bC7eW5WCst257Mls5yf85U8uKSQ5HAf3r7QQPjy34kGFE2PY+g8Md9afsy1XU5RgFmSJKmbyIwc6ezitIlCea+Ph50LIXcriq1vY3x7In9KruSr349icnAtT5oWE/XVFQwmHXe/GW0+KAJikjZ0LmSuE5kxW96CL38vOjmEDxarU4uuhoyfRNaO3g80enEs8UKRcrv6aZGKbasWKcZKtVil80StFxk++dvhttUw8jaRVbR3iefzczaJCWlTfZ5Nr4kCySDq8UhSD3G4RKSfh/vqO3bH2iLRsUTn2+rmcB8l6RWnmJFjiRT/LUs/tftLkuTdaoth98eej2VvgNpCMkpqmRClRnXh0yJooFBD3HjRZhxEMCfSQ4ZJkz4XQNE+WHSV2AbtHyMCQl7EXVOAomDn0a5diradOfO2QfyE9i+QPFMEbZQquPZD2PqOKJI89VGRqfPLv2DhFbD5DZFx3FgFmb+2f72s9aKjKMD6/4kC01oTTHsMCvfBhzNFtnJFJsSM8XyN8fejcDawcO0hHv1qL5H+Rp65ejBhFvHe9Mz5gYR/eXVLEAfE49/6thhfzOiW2yOGgN4XSZKk7iIDOdLZpaYIvr6r7bYJlwPjj38hSZOP4Z3JGHa8BUNugoosFHs+h7lLIX5Sy/lhA+HqdyH1e9FaPGa0mIj0vwL2LxVBk/RVYi+2QglqnUjnXX20q0PJQRGA6X8FjFkgivfVl4ng0GUvisnHsRQKkRK89ln48nZYeJWop+N2trQ+9aQ8QxQdVCjF5NQ/XqQwx43vpCdUkk7f4eI6fA0azPoTrGB7UlssJsrHFcqM9FGSW+um0XEK26Ms4YBCBnIk6VxlrRbZFO1QFu3j34OLCfNRQe4WEYjwCRPv25YIcZLbJYreTvi/thcwBsLUv4HBF8Y/IN7bG6rQuE/wXt4DKapyxMJT1EjP2YsOqwiujLil7TFLhNj+FD5IZOV8dac4zz8eMtfCqr+1zRzO3ig6V+n9REes2YtE1s0Vr4vaPMYAkamsUIh275c8C/0uEfMvl7Vl3vfzP2HsAkiZI44B+ITCBf8AaxWqV0fztwFlfHljHGpHPe+uy0SjVDB7RDQhVbvar6+z+U1xTRCLchf/R/ysJUmSuoncWiWdXWoK2nYqaNJnGtqf/iYmAqYg6Huh+DCXvV4ETobMgVG3iUmEMVBM0oL6wE9PiMJ2SjUkXSYKDn9xa8ubfcL5Ipiy9W0RUEm+HByNoqPD0rvBXg8BvUS675a3oGCH2HN9ZA3kbhY1O5JmiJWrprablVmw5j9iItTUMcMTQwCMvB1iR0N1gRj7TctaJpuS1AMcLqklwq+D2TggMnI8ZMtF+ChwuSGr2kXfgA52rlJpRbCzNK3j45EkyftpfU78vqq3YNb7iUBDwU5Y+VeYt0xk4SScDz/9XQSCtn8guk3O/w62viu2+MSNFwtBn94AjdUw/n4o2A0hSV63lVNpCRdBlMZKUKhEF83jC8Wvf1EESG78UjwfDZUiuBKSBB9fJ+YyEUPhon9B4S7wCRZzJU/2fy3O0xhFxs26F8TtpiDxPEYMhSW/g8tfhsK98O19Yr6ns4iFr76XwLf3isW0z24SGUFXvCG205bshx0LRbAIUH6zgITz/86fbSvInvwnMh1OrhwaieXQ557HBi3vR4kXwnkPi+wiSZKkbiQzcqSzi/u47RbGABg2X2TVJF6E4shqcdvV77d0Vdi7BOqKYe1z8MkceO8SccwYKIohNhVDdjlg3xew6gmY9jdxW2ACjLsH7HUiQBSYILJu1j0P294TQRwQE8Kv7hCrU047fDBDTO4u/o8ICH06F/Z81nrsB78Bhw36Xur5sYb2FxPD8nRYdI1IM150Fbw1TaQ811e0TYWWpG6QXlxLmOVUWo8XedyOEGkWb12nvL3KHCEzciTpbOZ0iG6Strq2x4yBIqPDk9ABIkhTUyDe73cuEvOH6hz4fL4I4lz1tljYARHcWP5H6D9T1GlJXQEfXQtlh0Vg5/uHxHmWaBGg8CY+ESIj0lYrgi8zXmwbjDIFiaydlY+Iuc+Uv0L6j2IeVZklzsnfDovniMyekkOi1kzogLbfrzoP/GLh2z9AxuqW2+tK4fu/iPo6Y+6BjF/E3K1p0c5aLQI/edsg5ej2cqddzKm+uQcqjsDXC0R2VeKF4ufZ92Iw+KE9+CUJ311PkqmWSD8D6qgh7T8flkgx7oueEYGkX58V46wuOMUnWJIk6fSc8YychoYG3G43RqN4Q8vKyuLLL78kOTmZCy6QdT2k02SJEJMle71IeQ5JFm0zC3aJbUrXfyaCI7Y60BrEfuxek2HQtWKVrq4MqrLFqr1PCAz/negAZTumxWTRXrBEiWCQb4RYDcIN05+GNf8WwaSczZ7H9+uzImV42f2QtkJMAm31IkX5eE67mLhMfURMVI780nIsdIAIAqWvEu2ZJzwgsoAOLRdBow9nwqxF4vjw+WJypJIJeNKZ53K5ySitZUiMX8fvXFsCkVFtbjZrFVi0cLjyVOvkREDpCTqbSJLknVwuEUDY8SEc/lk0HBh3j5gLGANEFk75YVH/zlojMmNBBCQmPSRqq7jsYh7RUAk5W6DP+fDuhSLAk7pCBCtuWg6VOeK20lTALYI8/S4RWSwqjZgH7PoYNryEu+9FKLysRg4KxHxpz+ci28Y/Dm74QixCVeWIr9V60TmzLF0E3cszPLcjdznF1vGQZPGzufINWPFQ6+1V5jBR36YpANQkqA+c95jokhU6QCxe6cxts693LhKt4Xd82HLbyFth9yeintGUh8Uc6cgaMb9T6UTW89rnCKg6gCEmARTDxe9JfXnbxzD5IRGce+/ilkWyNYj28jd8AX7RHXyCJUmSTs8Z/2R3+eWXc+WVV/L73/+eyspKRo0ahUajobS0lGeffZY77rjjTA9JOpv4hIoAR+5m8Ua75OaWY/nbRfrzjV/CqsdhxG0w8xWxcrbyYVAb4IrXxHnvXCAycBIvFBOD7/8q2o02KT4gjlXnQlUeaHSAEqY/6XkS06QyS0wYm+RuEgXzDq9qe27UCNH9yt4Ilz4nJhC1RaLLg95P7P2OGg6OBkj/QezfH3OXWClc/gAU74NDy8RK2u9WiCLMknSGFdU00mh3EeHbwYwcp10EKfV+Hg+H+yjJOJ1ATtr34sOFsoNbsyRJ6rlKD8Hb57f+kJ/2vVjsGHuPWOT58veizfi4e2HaE1BbIBZnvvsTLD9a8yagF5z3CFz3sehmFZwk3r/tDeK9d/ENYpGkLE0EIGqKxLkHvoVl/yfel3ufB1e/A6ufQuFyiuCAyYtqqjgdIpiz7V2xbdwYKP5m/vQERA6D6FEiY2fmayJw5bCK+oHtyd4gFs0aq0QW04T/E12mmsRNEPOvY424RSyobXgZ9n0pGlr0uUDUzvnxMSjc03KuyyECTyAW5sYsEPOtwkUw43n49MbWGVqpK0R2zpAb0WX9DANniGDMTctFlnRpqjhPrRe/P/ET4J0LxXNxrNJU+PFxmPEC6I6rfyhJktSFznggZ/v27Tz33HMAfP7554SGhrJjxw6WLFnCo48+KgM50ulR6yDpMtwxY1AsntNyu1It2oArNaIQXvxE8Ya94aWWrlDXfSwmeNV5Lfc78I1YvbnmPbF9qanwsNYEi6+HqY+JSeLez8UHz6BEMVlsz/HtSs0RYAxue55SLSYOn94gtnaFJMOlz4pJS58LxGTz4mfg6zuhprDlflnrRav0aY+LFTNTsFjx+uZemLOk/UlkQ6X4Pge+ERPVfpeI1TYfD2OTpA7IKBET5w53rKovAdztB3JMp9GC3BIhPhBU5YpCnJIkeb+GCrGI4alO3q//gQFXiff3MQtE8GXfl6Jui18MfHh56yyM8gzY8jZc8l+RadL3QggdKP4/d6sIOsSMFdt+rDUiqPHZvNZbNtN+EIV9Zy0CtRa8rNgxTqvYoj1oFgyaLR5b0R7xPKTMEdudmha4ghJFIMMc1v71jIEtP5vKbPG8q/WgMYgtUSNuEYtSTQJ6ido3H88W5zc5tFxkU1/7AXw0S/wtb75PPMxaKAJORftEJvbw+Ue7h3rYZrf+BVFXsOqYeV9IEtz0rcjQdjSCMUgs1lVmi1qHpiCRJf3LMy3ZQ/u/FNnTMpAjSdIZdMYDOfX19ZjNZgBWrlzJlVdeiVKpZPTo0WRlZf3GvSXpN9QUisnG4VUw8OqjabgZYkKQvUEEKWLGiK1GClqCOJHDRJDk2CBOk8YqUYSv78VwYKmYdFjCxZ74HR+IyVqT0lTxJq/WiWBP0mVixap4v9jm1G9G6/P7XyHOveAfYiWrvlwUSxx5K2x8taU+T/F++GCmaEv+4UyIHSdadh4bxGmS/iMMni0mH/u+Erfl7xCTXE+BnPoKEdD69T8tt617TgSMLnsJzKEn++xLUhsZpXWolAqCLbqO3bG2WPy3ne0I4T5KthTacbvdKBSKjl3bfLQYeHmGDORI0tmioVIETtpz8FsRvCneL96fL3lOfPC31YHOt3UgJ248jLgV3pgkPsw3mbtUfIC/4bP/Z++qw5u62+iJN0ndDWlxd3cZNhyGDocxYe7uzDemH9uADXd3d3d3arTUJW0aT74/TkOSNumgsA62e56nDzS5ubm5afJ773nPew7HrusO4jqdcdG975ZJB5yYDbR9GZAZS97/AMOmy4EINqpb0i+wRojtxBSuU8Xi2zOvAPMHU81y6Bf3O2w81lFzASRb+v5AIsYrkPHkDYaR3DHraSR9Y5criWOHIZ8eOLX6OPYZVhfQ51MtZUejIoJo59SS+6jUmqqs1PMQa5KBy5toVO0XRXW3d1Hto0kBlk90jOEBNFDuM43qq5x4NvkET0IBAgSUM8rd7Lhq1apYtWoVkpKSsHnz5tu+OOnp6fD19S3vwxHwb0LeTdjmDQL+fBSivd/SlHjRCKpSjvxKM+PD09k1O/C9a3EQ1dh1kS6OG7tJ9shUQL9fmKZQoYUrKWPHwZ+B0etoiJefys5RYCwLwJZPsvgQS4BObwEJ+1hESpWcsR63keTOiieA6ztc92vW00A5sjFQpRMLUk+4upWKo3xnEz4PCR3ZN1igKQOAlk+TvOn2Cce4ShsTEyDgDnAjowBhvgpIxXe53Ggz+a+Xn9u7I71F0JqAjMIyRJB7h/IzWDz+VoAAAf9e2CxMq6rUGhgyF7i2hRfoOz+hj073qbzfOwzo8QWw8glXEkeuJjEz4BdGUV/fBuz9hiqNC2s8P++NXYBYzMc/TLAY+PoP/MDvzC7vc0R7/UscC284gqNj9u9oo5aGw90+5Xl0RpUuJF2KUqMgkZH8WPEEjYgvrQUajeAo+OA/6D0YFEufI0+4sZvj5QDTPx/5iI23Sq0d25yc52iIOSOqCdD8CWDJGGDjqwyoWDiUY/XZcY7tjFpg6/sl68O8JGDNc3zvASqYH7b3V4AAAQ89yl2R895772HEiBF48cUX0aVLF7Rq1QoA1TmNGpXiFi9AQGkw6WDd9QXEaedcb7eagZWTgWELHNHeABBYFZD7OD1eT/M8T1D4UAUzaiWwYyoNj+0XmsVRoycLxD1fOW5LPgGcWgCMWUeTP5mSs/R235oNL9NAMaAySSJdjvt93zwCVG7v+TjtEEmA04scv4fXcxvjDKuZkejVuzPd68jvLHy8Q4BGo1iYFKQ/dLGpAh4c3MjQIqJMiVXp/NwVH0csQqR3UXJVrhWh6rskicQSwDtcIHIECPg3wcsfqNASSDrk/v7IxsDJ+YyOXjTcETKQm8BxqQbDORpdtSuVPfb7Y9oDjcfw+0gVBMzu40ikRJGnizLQ83EpfItUJm5CDR5k+FVg6paXHw2HV05mLWNH4iHWFn2+Z9w3AJxZTKXTyOVMiTJq6feXl0hPwehmvL3ZRNdmVFA1hlLEduT389h1VEn9VV0WXBMY+DvHZU16xo+PWkEyJvMKm18Zl4CIhoySt6P1c8CaZ12JOoDjtmufI9Gn9Oc6dH453EKTzOaeTAV0/4xKbQECBAgoR5Q7kTN48GC0bdsWt27dQoMGDvPVLl26YMCAAeV9OAL+JTBr0iE9s8j9nVYzF/KgqpQ+1+gFSGWALtuRTnBlE9D9U+DyRvf7aDaB8l+DBugxlakI7pQCIjEl2b+6IVsM+Ux3GDCdSp2WT1KJo0kh0WQxkTDxieDxuoNvNH1+vMNp9qfP44XumcUc/7KjWldKfgGObvX5wdVk2Q6LmUVTgxGMB7UWeY4YNMC2DzgaVqGF+2MRIOAOcCOjAPWj/e/+gdp0j/44ABCqEkEsAuLzrGgdVYYD84lgeo0AAQL+HVAF0NNmwVCOJ4fUIHkiUxYlP+5lWtWer9yTKqcXAqNXA2kXqEgFGCDgE0HvnTbPk3y4TeIU4cZOJlWdnOP+uJqMARTeRf4wD9HFvtXMOmf8ZjauWj8PyLzopXdyHrdJPUvSxE6UNHwc2PASkHQYCK3D+uPYzKLxNR+ep9wkEkPHZtEPUOoF1OrN9+XCKtZCCh/WLQ2G8TZ3aDaJRE38Hja/IhoAXd9n0tier/i+9fgcuLQe6Pg6sHQciRsvP5pRGzTu9xu3ByjMIpFj0jnqIncozAKGL6YHowABAgSUM8p1tMpkMkEqlSIzMxONGjWC2Elq37x5c9SsWbM8D0fAvwgWs9HV8K44CrMdnZ2GIzhmtf97FgoyFROhNCksGoqjWjd2hv7sRdO93zrwdnUoR6acEdMRSDjo+TgSDzJuXKrgPPaWt4GLqxl3fm07fW2aTfT8+PavsGBY8yy9cpaOAZaNZ1HU4XVuU70nI1a9w/haR6+h7NgOm41FrM3GoqzRKBoBuitWLq4hWSRAQBlgNFuRnKu7e6NjgNHjSvdjVQAgFYsQphLhRl4Zk6t8woEsQZEjQMADC6v17lUsITWB0avY2Fg4jGvkwqH8vUIzBgc4q0qK49ZpkhBhddj8Ca3FVEtdDn+/dbrkY4xaKnraPF/yvujmHJfW5QJS1d29ln8ahTkkSxYOB/7oyXO5aCSJl15OiuOLa4CqXfj/8LpU6thsVC4nH3eYDBvyWfsUZgI7PmFzacRSYNxmYOPrHDu3+8wY8qkACoihGqo4YjswjvziaoeC2f7eWE0kzlJO8Jhr9wPSLjG9s9FoEj4mXemv3f53p/AufWQqqgkQ2x7wugdrCJMeyEkggZgTDxj/4tgECBAgoAjlqsiRyWSoWLEiLJYyJo0IEOABRrESiuBqQOZV9xtENGD3BwBsVi7iqWc4Fz14JiPIc5OAhqOAphNonmcqBOoMZNd+5ROOfVnN9NpRBQFDF9DnxlRITxuZF+fw/wobX6NJYr0hgLEAOPAjUO8xxpx3+4hjVrs+47HK1UDt/kCdQSSjNrzsanJs1gN7vgb6/cx0DGM+i6c6A4CMy5SBt3yGs9wFaVTvpBxnB63BMBogO0erF0fiQRZnAgTcJRKzC2G1lSGxCqAipzhRWgzhahFulDW5yieCxuA2W0k/BwECBPxz0Odz3On4H/Syq/oIU6P8K/L+/DSuexIZVXuFWQBsbLhYDIyZTr/o2J/NRhWJTM1mSGmQKQGTlkRCmxeAAz+57scT9k/j6HTt/kyxNOTTE8a/EpB6jiTQw+ahIvMCZvd2HfW2j2N3eovj5gn7WadABHR8i9vEdqKPn83G4Alnf5ms6zR/Lkgn0XJmMTBuE1VNxWExAnP68bw2HEE1lNlAYsaQTw/E4ri+k8ohs4EK6cSDwOFfgR6fsckWGAPEdABCqnt+3epgQJ/LY/UOZ/2058uS20U14fjZvSA/FTi9mOEbMiX/VqRKoN2LpSeACRAgQAD+gdGqt99+G2+99Rbmzp2LwMBSZooFCLgDZGuNSNPoodF5oV7nqVAteazkRlGNOUZll9GKJY77bh5jtym4Ok2RL68Hen3NQuHEHM5L20eUiuPIryxk7AkNdQeR+CmNyIluyovGNi+wy5d9g4RQk3FUzXR5lwqYCi2A8VtJyshUTKja8hYvbHt9TfXO8T9c933wZ6DLB1T1nF1KlZEdcbuBao8A8wY4Ok1Xt7L4HLG05Py4MxTenl+PAAGlID6Tndhwv7v1yLHx7zeqSalbhavFuJhdRiLHO5wXg/mpgreBAAEPCoxa4PwK+pTYcXULsPtzYMJWEjRb3y1aOwOpGAmMBVY/DcR25giNM4njjHPLGEUd095zuEHFNiQZtr3PBCbnMWeblYoed40PkYgkx9oXgU5vkMDZ81WR+kPLRKXSiKAHEVlXPfv1HfmdY0wJ+xlPXvUR1hZiMX1zjv/Jc1KzD9DyKRoaF2bRB9DZX1Cm5HiWJxSkAtnXeD6rdmUza/Uz7hPCAJJlIhFwaj6VLlW7AjUfpSp7/YuO7dq/wmO7tLbkPtq+COz4mPXhqJUc07PZgMzLJOekCo64RzWhn2BZYSgA0i8BsDIFTJfDmrJ2P+DoTKD1FI9m/wIECBAA/AOpVT/99BP27NmDyMhI1KhRA40bN3b5ESDgTpGer8fbK89i8tzjqOQNyEKrwTpiKaXVAAmQZpOA/tOBm8fZvRNLAZ9oh8mwHZlXHJ2lwkxe3OXdZKSoJ2gzuW32Df7s+YodIrGMz1scMiXwyCc0DwQcRqsGDWXcV7dyfGtWd2DVk0DODV5o/tGDBWjGJeDyBkZrBlUhMeOMrGssGDe+BvT8wvW+ZuOBFRNLytQtJt7e+W33r1EkFjxyBJQZcZlaeMnECFC5Nyz2CF0e/zb/ooiN8BbhpsYGk6UMF0j2bmdO/N0/VoAAAXeEAoMJaXl6ZGvvMHq7IB1Y90LJ20NqUkG3eKRj7SzM5ljwpfVUYaQcL50UsJpJELd53mH+rwoiEST1Ajq9A6Sd5fiyTziQfIwmugAv2kUiKlEk8pL7bvkMCajUUxzpWjaOhMDxPwC/imwgGR6yMeW0857v02bQxDm4Or2IFjxGVcmiEUy5yr5BRcv+aWxa9Z7GRtdtBU8R7N45nmBPEctLAq5s5li8fYzLHRqMoIon4zJVXcdmAismlTTO3zcNqN0X6Pim428hMBYY8BtgMnC8zmLk+2g10ycxuBoTu5aM5r/XthapwcoIbTpw8Af6EWZcLjJWXslzWKHZve1bgAAB/wmUuyKnf//+5f2UAv6FsNls2HYhDT3qhuPVdsHwP/otZOcWApN20EzPagYkCqpj1r/CLtATu/ngm8eAVlNoZHdiDiXXAAu5Hl8Au6YCIbWYcBFY2fNBqENKmuXlxHEhbjIGqNyGXRVtBpMamk9iwXl9J8eklAHswLR9ETiziEUKwIKjRi8WDQuGuBY9duz8FBg0wzX+PLgax0Ue+YgmxoGxjoJXGeg+ghMgIaUMZLFkN3i0o88PvF2AgDIgLkuLCD8lRHc7uqRN57+lmB0DQIRaDLMNuFlgRYyfpNRtS8Cn6O86Jw6o1OruHitAgIBSoTNZcCOjAN9tvYJTSbkI8/XClE5V0TwmEEHeCs8PvHnE/ZrXZAyw6U33j7m4BhixBDj4U+kJixIZiYHNbzNgQCTm6JY2E7bophB5+bEmODWfSUxGHdDiSWDre4yqXv8ySYvhi4CzS4CUUyR8Go7gWPdBpzGsjEv0ilGHsJawWQC9BkDMnZy+BwP2ppg7eIeRBOv5Bckz30jg/Cr3xHjWdTbG6g8pOQ5lNnC83C+a2xRHlS4ARLwvrDZQvRvrlbg9JZVXLZ4E4vdSge2MwiwqiGr15dg8UJRo+gTQ8W1g3EZ6snn50GdHIgWGLeRY1v7vqYze+TmJOjvyb1EZ1ONz+hp6SFeEzcamhNQN+Zef5pqmevuc6DnSV7whJ0CAAAHFUO5Ezvvvv1/eTyngX4ik7EJEB6iw6Ggi2lU6Da8TvwNP7GH05I3djg3l3vSO8YvmQu48jiSWcqGs0IoXjpXbAvu+ZXGQeRV47E/O1KuD3UeNN3+CkeLFcWUTzfZ0eUCf72jYqEkBCnMd40tmHbt6YgkQ1RTYXTR/HdEA6PIeCaC8m+zQuINJxwJIInOYA7Z4ivLygjR2lfp8z2SJC6upTioVNmDiduDGLh6/f0WaIPtXePjm+gU8MLiRoUWoTykXbZ5gHwtU+pe6WbiaBFF8XhmIHKkXL0QERY4AAfcdJxNz8PiMw7AWieUyC4x4av4JTGwbg+e6VIOv0sOFrycTWqlX6QqFnHg2R/Jucowp7VzJbeoOYsiA1ItNlFVP3TbzFwFAheZsXhz5jaqL8PpA3x9pkGsPRdBmAElHOGLT7VP6vGx5x/1afXEN0OsbwGykIre0QIYHEf4V+R3p7ry3fg44s4R+N0PmciToyibP+zq/AqjU2j1Jd2YpUzgXj3T1/wuvB3R+F5jT1zV0oekEoM+P9Dm8sQuQq6jEKUj3nER4ZQO9B+1EDkCS5dZJoEZPjpFtesP1Pao7COj9Hd8/ZxLHGTunAjV7s1ZyhlFLkvD4bDYLYjuyQedf0eHJdm1rid3dRtxuEo33CTab7e4bKgIECHjgUe5EDgDk5uZi2bJluH79Ol599VUEBgbixIkTCAsLQ1RUWXJkBfyXkKbRw2CxQiwC3usYhMCl31ESfWyWK4kDsNNzdglQq19JTxmrmR22J3ZzlnrpGHpljF7N4kUsZfE2cjnHj+xGyhI5OzBevkDCgZIHKJECtQdQoj1/CNDxDWDDqzyWQbM4xhUQQ/JIHUL5L8C5664fAItHcduGboz8XFC0KMtUlIrnxDsUOMvHM63KZuHr8Ymk0seQX3I3MiW7a/4VgMajgIbD+doFCLhHxGdq0TK2DF5o2gySlH9BQAYqRZCLgbg8KzqV5QC9wxidK0CAgPuG9Hw93lpx9jaJ44wZ++IwokVFz0ROhZbub/+rNcnLjyTQ7i+Ax+bQ4yTpMO8TiXlRXr0HsHwiMGoVsGh4yfUw6Qi95tq9QmVu6hkSQl6+jhErgBf755ZTnXNhtYPEkakYWhDbkRfr+WlUjvhFARrJX5q3P3AQy9jQWveiw5NGIgeajqcXzonZvE0kIkEjLmWEViJjgpe725uOY4pV1w+ptsm+zqaWXA1seovNLVUwlTYn53JcKqIBcHY5EFCRyaKJBxkQMWKJ59fiLpmzSlfWSRteLulhdG45/QN1uZ5fl0FTRDI5ETkmHXB5I+tG+z6vbOLf5rhNQKjT+L8nSGTuR/juEmkaPc6naLDq5E14e8kwtGkFVAxUIUB97/sWIEDAP49yv1o7c+YMunbtCj8/P8THx2PSpEkIDAzEihUrkJiYiDlz5pT3IQl4iJCrNcJi0CLKlgOVOQvBUj8WZ7EdGI/pDk3GAds/8rzTYzOBBiOB4Qu4sO7+Ari0jkRP1a7AIx8zncpiom+NTAVYjcACN1HlITWA7p9Rdiv3Btq+xG6OMpCL/Z4vge6fc5G32RhFbh8fqd2fCh9jAX/X53FUKv9WyeeRqxmLOmgmi9RTC+ifY4e5aMb71hmmVo1eww7WsrEl9/XIR4DaaXxKIHEE3AfojBakavQIL3P0eMBfpkmJRSKEqUWIyy1jBLl3mKDIESDgPkOjMyM+q9Dj/WeT8xAb4sFE3yeMCpiTxWrBnASqZuP3lXyMwpcqG/uoc/oF7uPRb6mGkMgAmwg4+SfX9MIM900NgClKE7dR5ZF4iE2X3V/S0Nd5/Fgk4ghz72lU3Cr9uc6fXULiw6ABqnfnuM+JuUBUI9hCa+Gh0kQYC2hSPHQePWrsI0LXtrPGGDYfEEn52rWZQJ3+VCjZoQxgqlNhFuub7BvA4ytIlmmSqbhpMIxjRDd28mfoPI5ihdTiua7Zi+NNuYmAbxTPZ52BJHS6f0YzZOf6x6BxVSrbUWdgScVQeH2mV51d5tmI+tgsjuGVBmkx1WlBOtVexfepywHWPAuMWEyj7pqPknB0hzoD73msPTVPj4mzj+JcisMCYMHhRIxvE4NnO1cVyBwBAv4FKPcrtpdeegljx47Fl19+CR8fh8FZr169MGLEXykQBPyXkaM1QpNxE9Fnvofk1DyorGaSGDV7s7gweSgc5d7uyRA7NClcYINiSXrYCzWFD1C9J2fdc5M4YhVSE9jyNtB4NFCxFbtAdsR25LjV8okOlY1vFFOmIhuyoEw7T+WLQcPZ+vxUjlcpA5hotedrx/5OzqXfz/7vgaRDlOra0fUDFgtLRnt+Xfm3uN/MK+w2DV8EjF3PzlfmZSCwCqXGkQ056pV9jX5BJh0LjICYe0tkEPCfRmI2P4/hvnebWAUqcu4wrSNcLUZc3j0QOe5ibwUIEFBmiP+CrfCSljIGqQxgemNMOxrb+kQA7V4ikdP1A2DJGJIAdki9gL4/0GAXKEo28gZWPc3UqcWPFx2UlOPGUU2B6zs8P7/FSIJGGQiE16XSpMWTrAF6fEbjXLEUGDqfxMSSMfSn6/EZELeXF/V9vmcdsf1DIH4/yYmMy/gH8kXuDYVZjhh4ZQCw9xtGdz/yIXB4Ol+fzcZmVIfXqIy5uJaKmkc+ok9h5mWOE0U0BNY+T8Kn5qP8js+6Rs8cewiDTMW6o4qE6uj4PXxOOzTJTCxr+RTNp5MOcazNGd5hJUmcoKr0WNr/A1XQMiWPL6QGR97zkjyfg4JUvqeeRuwrteV9zkg753mM7uYRKotUgYA6FOjwOpuHzvCNoqm2/K9G4j3DYrVh2fEkFxLHjln749C3YYRA5AgQ8C9AuRM5R48exa+//lri9qioKKSmprp5hAABRKE2DxEnvobkzHzHjTYr59DrPcZiITex5APzkmg2eHWz+x3X7E0PnatbHCSORE4z4Z2fucZyS72Agb8D+lwutFnXGVkuknCEamY31wVckwwseZwkypXNJEwM+SSJen3NTmJ+Gjs+KadYgBZmAp3fAwJj+NpCagKtnqFyIPEQZcgn5rCAcjY0Lo7weuw0ASSQDBp2NIcvJFkjVQKqACZ/HPwJ2OtEIu2fxjjRfj87TGEFCLgLxN2OHi+DIkeb/pdGx3aEq0U4kVbWCPKiDrtJx+JegAAB94wAlRyNKvrjZGJuiftkEhFqRfqWvgPvUBrjVu1KAmTeAH5G/aKBbh/TNDj7BtMbo5uRLIlu5hgxXvs896PP5VpuMfJ+gEpbv1JG+O1mycdnMRih4Qg2aK5tp/r3qYNUy15cy/33/YHEwe+dXFMho5uyVlgymttWbg+bSAKDyQIv2V36ef1TCCwyZj42i6reOgPZ2Do6E/CNBkYsBY7+ztpm81usa9q9xPuWT2ATyQ6FDzBgOrB3Gkm6I78WkVtFkHrxfpmKdYnF5Goe7YyjM4BRqxkS4YzIRoB3OPdzYQ3/ZuoN5t/NqQWMna/dj1Hmx2cxkSqkFlVB51e6f67IJhy5GzofmD/YVcnlXxHo9xNJrsJsvv8KX88+T3ZYzdzm5mG+ziFzWevpc4HYTvTsCahc+j7+ApkFBsw/7KYeLsLCI0loEO0v+OYIEPCQo9yJHIVCAY2mJEN85coVhIQI3X8BJZGtNSBPZ0KYNRfyc4vcb7T3G/rkrHzC9XapF6DwY7fo+raSM9I9vmDX48oGV3+dugOBcytcSRyAaQIrJgGTdgL/a8XFNrYjzRWP/1myC+NfiZ0giYKz5td3cqb+2Cz+hNXheNSOqUDrZ0lMyZSMWd3mZAx+6Gea+dUfQhVPi8lAymlGZ65wE3UeGEtyyW4aC+B2N9Bi4vPYjzX7hiuJY8e1rcCVjUCTsfzdkM8CViSmh1BxObEAAU6Iz9JCKZPA16sMy4w2g5+dO0C4WoyUAjMMFhsUkrssSu0kZW4SJfYC/hnYbCTUrGZ+ZxfvcAt4qBCgluPzgfXx2K8HoNGZXe77fGB9hJRmgG7IJ1EjEgEQ07vOfmGcdxNYNp4X5lW70nB4w+ts1si8mDblfKEdt5vrZmx7hhhYrfSPaz6ZjY7iUeUVmtOnxZjPesJqodfdvIFA47FcUwuzqBS6to3raM3ePMbia//NYyQ4avam6q/pOEAsgcWdcdCDCq8A1je6XHoEJR/nuu8XzWbS/u+oPjIbaDp84Eeg+1SOCzmTOADfl9VTOO626ilg4G/8vMfvJ3kWUoM+OH/0oEJ69GpXYswZ9hrG2bg4oiGNkbOuMmWsziA2owIr0QNRJAYOfA9U6QRbdHOIDk/n49IvkBBUhxSrl8C/wXYv0tT58P+A8Vu4/6zrJI1CavD76uJaqqkLUvk31OENz+c0IIZNCk0K1WJWC0muql3ZWDi9iOP4bZ6jWruMsNpsKDR6bnBodCZYbTZIBCJHgICHGuVO5PTt2xcfffQRliyhIZlIJEJiYiJef/11DBo0qLwPR8ADjqTsQvyxPw6LjyZhxwh/qNyZ1QFcjP0rskg4+BMJj8jG9JKxWSmJHb4E2PS6w7Svy/ssynRZQExHIKIRyZRr24DK7dwTJADJnKTDVMpk3yCB0/q5kikZTScAFVtw/nvHxzQcbv08ULE5cHI+YMijUuboTKDJaCBhH1BnAM0Rnf1u7Di9AKjcGtjxKSM5g6oBFVsC/f9HiXN+KouVat0pTV8w1PHYiq04YnZhNf2Csq6R7On0lufYTIDnsnpPSoG3fQhc20IlT6PHea78oj0/VsB/GvGZWkT4ed19x89i5IXDHY9WiWADkKSxomrAXRa+3uH8NzdBIHL+KRSk80Jo7zdAfgpJ8Uc+4ujEHf4NCHjwUD3MG+ufbYdN51Kx/1omKgapMKJFRVQIUEHpTpFiMXMkZ/dXwIVVHJduMBzo/wsv/O2GwiIRGwon5gAtn+aFdvYNjla3eobkybGZJAcV/kDLESQQ4vdS+ZpxmevlwN+Z6nh5Pbft/z9eyC+fwLHk8PpUxkpVwMAZHHHeP42KiwZDOS61+ysqVDyN0Zxdyv1qbpKE8qsAdcBD5EFntdADyG7ufGwW1ZKRjYDun9Lrb/3LwOA/SORkXiEp4ymNSZfDmmz4Er6PxgKOdnuH02fHVMh9KQOAv3ITkki5rT6X3xOZV2kuPGwh0GQ8ibq9X3H8ShnImqX9a0DmZYj2fee6ry3vUsWz7zuHB1NADNDxdaqgVYGMJV8+HhizjqoegH+Huz4HDv3i2NeFNVQkNZtEtZIzRGKmYPmEk/Sy17OGfFdFkCaZf2POBtt3CT+lDF1qhmLFyWS39w9oFAWJ+CEb9RMgQEAJlPuK8s0332Dw4MEIDQ2FTqdDhw4dkJqailatWuHTTz8t0z5//vlnfPXVV0hNTUWDBg3w448/onlzN+74xbBo0SIMHz4c/fr1w6pVq8r03AL+PtzK1eGT9RdwPkWD8W1j4O9fbAOfcHbWKrWhf4xBA1Ttxg7S2ue4wNpRrRvQ/hWg70+O2G+Zihdw+75zxH+LJSRT/CuUHhWq13Be394pzLhIYuTmMf4e3QwIqw2scFIIaZKBTa9Rqv34MhYgVzYDpxdyZvzWKaoQTs2HR5xbwW2PzQKe2EUZ+bFZQOd3WGCKJSSCNr9NoiXxIDuSYgmQeYmFhE8EiZzsG/Tzafsi5cf2MSyX15lHwun3To7XaizgfPy1rcDotaXL1AX8ZxGXqUWYbxnGqgqLfAj+InrcDnsEeUJZiBx7Op1gePzPQJdHYvrEn47bUs8Ccwcw4a/OAKbjCHjoIBKJUCFQhUntYzGqVSXITXkQa28BSTepuPIO4xpuR24ilQ5VOgD1BwOp54B933KkqedXNIlt9zITf7QZvDi32YDlk4D080VPKub62nsasG8a1bCaZJI4gMM7RZ9HFU39YUWplTo2VJxHdW6dBlZN5kjNn49SPQJw3T78K5B4mGtn8jHPJ8FUyLqg8Tjg+B8Qdf3wPp3dcoJERu+8IzOAS2sdtyefAJaOJcmWepZNJJ+IInKm0LNxMMCmUOJhYN83HPM+u4yK6WXjqJoCqHJp/Rzfx3UvlNyHXwUg4wrrPKnCodzxr0ilS3RTYMEQR9R5YRaJuMptqWI+OtN1f9k32LhrPJaJZSIxyaTVTwGhtR2ES/pFGLS5yDarEerjBUlBuiuJY8fh/wF9f4Rt4EyI9n9LYjCyMeu04KKGQY7nsScUZrmPab8LqORSPNu5KjafT4W2mDKnZrgP6kf739P+BQgQ8GCg3IkcPz8/bN26Ffv27cOZM2dQUFCAxo0bo2vXrmXa3+LFi/HSSy9h+vTpaNGiBaZNm4bu3bvj8uXLCA0N9fi4+Ph4vPLKK2jXrl1ZX4qAvxm5OhNqR/iiS60w7LiYDrM4CPIqnSGK3wcM+I0dmeOz6W0T2xGo0YtxoQe+d8SO2nF1C4uSgMpMTFAFMj1hx8cs2OywWlhY1Bvi2XMHINGz4gmSR17+wGOzgcrtKcENiCka8/Kg6Dm1gBcoC4ez4/jIh9xPpTbs3NmTN9xBn0cCymblTPbeb4CbR/lTHJ3fIemyeISjgJV7s5OmDnZ0gA7+DAyZ457IqdIFuLHH/cx31nUa9/kN8Hy8Av6zSMgqRMvYoLt/oF3efoceOf5eIigkQHxZDI/FEnaQ7ebkAsoX2nRXEscZm98AKrWk8aeAhxpeunQSMc5KjcBYRkUHV+Nasmi4q2dKpTbA4FkkDDTJwIhFwLYPXEeOw+rQM2flE1Td2qxU2cR0pNGxOoiNDjukckeikUnHZKyYDhyvLu63AjD5asfHDhLHGbdOFXnvNPX8woOqUnUbXBVIDoeotLjpBxGGfI4lOZM4ztj9FdB6Crer9ghrMNhIguhy3D8muAZVKwBHiNq/Cswb5DrWZNYz4bPX10BsF+DGdsd9Cl+Oqa9/pWhbg+P2QTNZI237wD0REr+PjbzwesC1NNf7CrNJHMLG+jG2E+vBOgNJMgGARIaLaYUYv2of3uhRE4Nlx9zbV9tswOopSHz8IJJaz0LDSBW8VL6Qqv0d21TpCBz9zf05imhIMuseUTFIjbXPtsUPO65i24V0KGUSjGhREcOaVyibd50AAQIeOPxjGs+2bduibdu297yfb7/9FpMmTcK4cfyinT59OtavX49Zs2bhjTfcz6laLBaMHDkSH374Ifbu3Yvc3Nx7Pg4B9x8FehO0Rgu+3XIFK4YEQ71oAGXKj3xMAuP8CsfGiQeZ8jRkXkkSxzuMc++htVjExXYC9n7LYs6ZxHHG7s+BLh9QSlscEQ1ZpNgJF30ucH075b0TtpJMshodUnB3yI4jmXJyLos9VTBgNbHLVLmdozNVHDHtHaSN1MtzN9A3sig1o1jsurGA0agjFgOX1hcldBh5LkQi106aTAm0fQGY09/z6zi7jD4FQmS5ACc4osfL4KNUYCdy7mysxh5BnqApYwdTHUqPHAHlj4xLnu8rSKdiRyByHm4YtVyHio/bZN+g98yYdcDCoRyNcUbCfn4HNB5DrxuLiaazzkg7Tz+Uti8xiarJWI61mHVFF8lS2ORKx5DOhdVAk3HAkaILaKsF8PJlspI7RDQourj3gIT9HO8Kq1tytBqgYmf/NHq3NColYfJBhdXs/nXZkRNHFbF3GKC5RdVL1a5UvWx8reT2sR3ZHOv4JptrUgX/Dop709ix92uaLF/bCmReo5dNUBU+rufnrFnMJk5h2azAhldZI5Z2zNe2M/Xq+o6SZI/ChwTivu+AKp35d3V+5e1GlqHmICy/ZEC21ojvtl1Br+5SeJdy+nQGAx5fmAKpWIRFT7REU7XTnRENPTYLrd0+QZbFG7Z8PUK8FWU2JJaIRYgN8cbU/vXweg8TRBAh2FsOqURQOQoQ8G/BP3L1tX37dmzfvh3p6emwWl2/SGfNmuXhUSVhNBpx/PhxvPnmm7dvE4vF6Nq1Kw4ePOjxcR999BFCQ0MxYcIE7N27t9TnMBgMMBgchmvujJoF/D3wVcowc18cvns0EpGbJ5FY0GZSpupM4tih8HOMZdhRoSXw6DfA9g+A1dtIVEQ0BHp+4UiocofkEyR+Bv7OjlxuIjs5dQbQCHn5RNftTYUsALa+R1JpWCnjUQDTqUx6/v/wdG5/dAajUWPacba+eEfLOxSo1NoRySpTkkApPgLmEw60eQE4tdD9c9us7IhV6wZcWsfb/KKBho8DZxaTUKrSBej2CeXrpXnoqAJpAFkOED6LDw/uOXpc4VP6310xhKnuJYJcUOTcLe7bZ1HuU/r9d/E3IOABRUEGcHaJ+/s0KSTsipM4dlzZxLXxfK7npkvaOY4yazPphaPPpfK00eNAZCOIVEFcK32jePHe7uWi8Zrf+RhdLhsp7mA1c1/GAvf3q0Kowh34O0mHi2tIOAXEML3p5jGHh46pEFaZ+r4HkP+t66JYXDqhLhKRxJnZ1aGMSTzIUane3zFWOz+V57/BCKBKJ44idXkfGDqX5z/HQ+omwMeaCoFjf7BGSdjP9+Lx5fTkOTyddVJQFfoPVmrNBptddeUOEhlJpKHzgE1vOr77IxrQM9De/KrWDdj7Hb0LASCoKhIbvIAlc+IBALfy9EhT14S3WOqq2KrQAmjxJGwyFaIlhVg3uhK+P6zBu6vP4ecRjVExUEUixS8KGLMW2PQWQyVsViCgMszdv8DKW8GYOncPvL2kGNcmBo/WiyjbmHIRVAopVIoyXu4VZnPUy1hA82vv0HuKRRcgQMD9RbkTOR9++CE++ugjNG3aFBEREfcUfZeZmQmLxYKwMNd45LCwMFy65L7Tt2/fPsycOROnTp26o+f47LPP8OGHD9lc878AFqsNSTk6WKw2NAw0UXLd9yeaABf/m/EJ55hUTnzJCOFHv2F0qbM65tYpqkzGrPF8AGIJSSFtJkkRVRAQUIlky+JRJcmTtPMsJhOLCMTUs0wvKN5BBGj2F16fsvCE/SSmJHLO8bd8EjjyOyXlx/+kakYkpuql0ePAupcoIR7wK4uHekOBU3O538BYFkj6PHbHqncH6g1iYVI8ojw3gWbJANU7PuH0Iej4Jl+7XM2LaauV0atb3nF/npqML/l+/E0QPosPD+zR42G+ZVHk3Hn0uB1hahFOlTWCXB3KNBYBd4z79lkMquL5Qjm6Ob93BTw8yLtJb5v0C1zfwuqwMeBuNAnge59/y/P+bFZekDceDSwc5nm7gnQmO9rJBLuPW4Ph9NEbvphKVt9IEgKZN2iKm5/qiNiWqUgaOEOqYDT10Rnun7dGT2BWNyC6CUMABhZtp83g86cVefd4h8IWUAUWi/m+Ezl/67ooljEowR7hXhxVHyHZVjxdaveXQKW2jCfPSwICq3C7pWOBhiNZ06x9nr6FIbU8P789vS7rGn/EEmD8NqZ8Oo97ZV2nX06Pz/ie1hnIplRxiMRARH1g0Uig1gCShHZ/tMyrrK80ycCj38Em9QLq9Aeqd4MltB6S9AoMnZ8AlVyKx1tGQSETY0O8EU/1+hbSdc9xHzV7829i3YsQ6XPhDaCuTIUvO36K3zLq4kpaAUQiICa4SMcTUBkY+CvrTKsJ+TYVRi6Ox5mbrNdyCk34aO0FbD2fih+GN0KITzmPQ+Uk0Gg8YT9/l8hY87V/mQSeAAEC/nGUO5Ezffp0/Pnnnxg1alR5PzXy8/MxatQo/P777wgOvrN40zfffBMvvfTS7d81Gg0qVKjwdx3ifx5Wqw0JWVqsPZNye7GTWIuKBO9QkiXqEJrd+UVzIfSNAhaNYPHY9QNuc/Mo4yfj9rgfcTLreHtIDde5fDtq96cENzCWBQIAdHwDSLvgvqCJaMDZfDuOziDZsvJJV9mwRMZkra3vA/WH8jWlnqGyRiIDVj0D9PqKo1cVWpBE8Q4jqXR1O0edIhqQrDm9iIkdSQdYSPX8kouu8/OpQ5jGsPZ5Ftl2hNZiYSRX028ofj/TI7QZ7CL6RnH23ycCqDuYyp3EQ66vud0rPP/lBOGz+PAgoSh63E9ZBkWFNgNQ3l1aUZhKhFtaG0wWG2R3G0HuHVbUcdTy8yDgL3HfPove4eyML3jMtYOuDgH6F3mZCXg4kH4JmP0oL0rt8A5lkyWwChOpisOgKT2ZRyLj51MdDJi0nrdzNry1QywFavWmp054Pa63hVlA0lF63Enk3O/1nfy+GTQTWDLKQTpJvQCRlMqMhAOsL5zR7VOOcVvNwOVNgMVAA+XiCKoK5CZC5BsFiUkP+Hr2biwL/tZ1USxl7dHra5oOO48i+UZxZP2PHu4fm7CPpIjNwhH1phPoh9R0PAARa6ysOKBpRxL3+tyS+2j1LBtXE3fw/HoFcDtPnj17vwWGzGU61c2jrg0skQjo8QVJn/GbeAwJB0g2XljDdKvQ2lRXZd+A6Nd2t/+mpCIxQpo/jy3jR8PbmAFLwhaYlcEwRTZHgaoX/J5sDNHxP5lmNaef63kyFSJg64sYO3Q99D4WeOUnwGoRQaz05/efwgdQ+CBfb8Lzi07izM28Ei/r4I1sxGVqy5fIKUjjyKPzmL/FBBz5FZB5AZ3e5udOgAAB/yjKncgxGo1o3br1fdlXcHAwJBIJ0tJcR2TS0tIQHh5eYvvr168jPj4effr0uX2bfbRLKpXi8uXLqFKlistjFAoFFArhy6q8cDktH59vuIi+DaNQJUQNpUwCrcSPRYBUwXnlzCssCJo/QbO8w9MdRda+71iQ7f2GZnL20SF32PERLyKWjaeCxo4aPSm/ntEV6PcTyYz8W4wM7/KuaxIVQFIppj07yJENmUKlzeS8du/v2JG6dZbd5+hmJJCqd2cR2HAkYOjD2f1hC6jgub6DPjkRDfiaNSnAzk/ZTTqfxuJEGUCiyGICHpvLInf10yVnzbUZwKY3mABhn1mXegF1BwE3T/Ac7viEMvBl40kq2SH1AkYuASq0Ah6bw/N+fiULj3qDSaQpA8r6Vt81hM/iw4P4rEKElyV6HKAB7l0ShGFqMSw2ILnAhsp+d0vkhPDf3CQm4gj4S9y3z6JUxnGIZ44AlzbwO6ZyW6oA/AWS9qFBfhpN9Z1JHIBr3fIJQI/PSdYVR1hdXix7Uq82Gk0S6MYujjW7M+SP6cDxj+Ko3Q+4toMqV2UA10f7Ot/xDRr1H3GKh67alQbJ8XuYKFSpJaBJYqpaj6m8OE84SE+d2I58zfb6Iukw0O8XoFZfEhinF5NwCKvDBsuy8cDQeRCnXShK7Aq5g5N6Z/hb10WJDLi4lgreEYtJfBSkcTzdO4xEjb6IePAJZwqYXzRrltMLqYyK388o7ootWSttfI31XGRj4PQCYM/X3PfyiayVACpvmk4AgmKBHxvztsjGVNzoSxkd02YAEhlsxgKIOr1FE+abR6nsq9GL3/GXN/C9PDydHjW+UayHwmoD2uwitdCzrn6BNiu8D38HdXh1iPZ84VDxSGTI6j0LyVFtEd1tKqyb34TYncmyXwWEqsTAtskQ2RPUvMOA7p/x707ph3y9Gbsue/AKArDhbCqax5SjQjEv2bNX45HfgKYTgYCK5Xc8AgQIcItyJ3ImTpyIBQsW4N13373nfcnlcjRp0gTbt29H//79AZCY2b59O6ZMmVJi+5o1a+Ls2bMut73zzjvIz8/H999/L3T3/2Fk5htwLD4bbaqF4LONF9GxRgg2TqwBX4kJtqjGEM3p59g4/xbVNn2+54Jjhz6PRVPrKUBsZ0ccuCekXwYajaL6RJvJwiX5JHBiLjBqFdMP+v3EYi7lBIvNHp9z9luXQ1VKcHUaIuYkUOny6HeUEZ+aDyx+nB25dq+yKInfywJHn0eyRuHLLtCKSTRejGrCQm//d5T6TtpJj57iqVS6HKpshi3g6x08y7PPQOZVx0VRQAyLUrkvi6jUMySTzi5xJXEAJkcsGAo8fZhjZT5h9O8RIOAvEJ+pRahPWS4ubPTUiGh0V49yjiCv7HeXwwt2iXhuokDk/BOQKqh8bF1yzRbwkECbQaWDO2RcpuKl6QSa+9sVrbGdOVqzYhLQ+1tGQl/bygtoiQyoP5xmwWIxx6ZaP8uL7HMrqIIRiYDqPdmMcEcC1exN1WnGVSD1tIPEUYfw723X567bX9vG0ILhi6j8UAZyTdbnAqueZkMnvB5JjaMzSAJ0fItNHJkKWP8iCYyoxiQmpF58rV5+NMu1mklQWAwlDvWBxZVNPF+HfiEBEt0cUPqTBMm+QbWvTwSVTxVb8bycmseap9vHJF+ubAZ2TeV5qD+M9dOuL4CrmxzPk3yMhtBBVanS8okATswGljgZRKecAOb0ZaCETOk+SROATark/Xu+4jYhNalwPvQL30+RiGPqeUlUaSUfZVQ9ANR8FMiN8xifLjr2O1B/CEfHAMBiQtDacUgauReZklAEZ3ggPrp9AtHyCfz7sKMgjUqlkcuAao9ABEAuFUNvKkkEVQnxxuCaChiykmAWSSDxCYWX7G++fMuO83yfSQeYPPhGCRAgoFxRLkSOs+zTarXit99+w7Zt21C/fn3IZK7S+2+/LSUhwMO+x4wZg6ZNm6J58+aYNm0atFrt7RSr0aNHIyoqCp999hm8vLxQt25dl8f7+/sDQInbBZQ/dCYzjBYrpm64iP51AvBazUyErp3IKO9Nb7p/0Oa3qExxltoaNMDOqYAymAv2idnuH9vyGaZTtX2BJomZl7nQ23FuGTBmPUmXHp+zWMtLpCJl0ExeAF7dzOhTO+IySDA9+i2LSLv0OrQmJd43djntfzlweSMwfjM7gWnngAM/0q+jQgs+h9UMtHmes/uHp9OE2Y78W+xmdvu09MhygKaiE7fTMPLaDpI0duKmend2Td3BpANSTpLIESDgDpGQrUWzymUYi9Frii5+/O/qYUFKEaQiICHPCtwtH68M5AWWYHgsQEDZ4OGi+jbyU6nOGDyLI00BMVRJ7PiYqo6l4zhy03QcVaZiKddDmZprlTqUSthGjwND5nBdlMi41h75jSEBxSESUyFr0gE7P3bcXrsfDYrdwWYDDv5CdcbpBVTZ2D2c8m+5+vmE1iFJkHjQ1UPnymbg6laSOXoNX2e3j2mqHFCZvjMPC5JPMNxBLGFtVDwRND8d6D8dSDvLhpId2kyOYnf7lPVTQTp/dnzMCPDq3V2JnOwbrKPavkgV1vmVJF6Kw2wADv5Mku+4m2CUmPYQSRVAxkWmji4dXSIVytbiGYiubmV6aa0+wK/tHXeqgl1H0IsjL7mkMbbVDGXyfmSrBkMVVBeqhAOu9wdU5niaM4njjK3vApENEeQdhMeaRGPuIdfjfa9rBPr63UDw1leBzKtQBFVBfqvXYYntAHXA/R3Tc4F/tOf7JHLWwwIECPjHUS5EzsmTJ11+b9iwIQDg3LlSIgLvEEOHDkVGRgbee+89pKamomHDhti0adNtA+TExESIxULU3sMAo8WGX3ffQKBajjebiRC6+DFKYOVqz0SFUctFxR0i6pK46PQWsOsz1y5Lzd6UdVsMLDqKz78D7DKemA3U7svict+3JFi2f0gVzmOzuV932Pkp0HsaULULULk9i1VnEscOU6Fjf1c28THJJwG/SGDjqw7JuDqExeDFtTRAtqMwi53A4UtY/LozlRRLOXollgLrXwIaupHAe0p4AEpP9xIgoBgMZgtu5eoRVpZ5fm2Rn9VdeuSIRSKEqsoYQS6W8ELRLusXIEDA3UEd7LjYLw6JjGv0mcX8Ca3FyHF1EOPCj//BdfDgT/yxY+J24PI6jt70/ZFNk+N/8scZQ+fTMPfEn64X7SY9jZalXq7rm8K35PrnjMJMbpNxicc9+A8SAs5kVWgtjivDBhybWXIfNiuw4TUSEz7hQExHIGEfbEFVIfJ5iExiYzvy9T36LbD+Zdf6omIrqoxFIiqU3WHXVDbarm5x3HZjJ9B4lHtfnDNLqJC+sdPzMSUepA/O6fmuvkh+FaiQTjzIlNJ+PwOjVvP4k08APuFIa/gsrkliUTdMCbXYCOnlYqP32ddJCl7f7v65w+u59XqSF6ZCJJYgq+bjUJ3+w/XvLbCK6+h+caRfBEx6yNViPNmhCvZczURCFk23+9QJwiDxbvit/8CxfeZV+KydCHPHd5HTaBIC/O5urbxj+FVktLy7BkfDkfS/EiBAwD+OciFydu4s5Uv5PmDKlCluR6kAYNeuXaU+9s8//7z/BySgTLBYrEjPN+Dl9uEIOfIxiyGpF6W8pUEZyJ+6AylrNunoOWOxkjyRq4ERS4BbZ2hyHNWEyRp5yUDF1pRUe8LF1Uy1UAaQaKkzgNGXeSlc0D0RIIVZJF/kai54Z5d6fo7r21m8HPqFZokTtgK/tHAlnrQZwOpnKPu+tp1dSoDdwoJ0FrfNJwGH/ldy/41HA9d3sds54Fd2iEQSdiuPziRZ5FfB84VsVBPPxy5AQDEkZetgAxDmVwYix25MXgbvpTITOQA/q7kCkSNAQJmgDgWaPQEcdrf+jGEstx2xHUmUABxx6vU1fVOcvUU6vcXmzZpn+buXP9DhTWDP5451USQimeIXBVj0wMilbNxk3+DYk3fR+gsRfVtuKy1s9GG6dcr9a4luxjGxofOpEhGJqI41FnKNDKkBiOUkIbKzPY7hICeO32M2K8mhwBhAIoXFaoNEXD5Jj/eMyEZsSimDgCf3A4lFHjnRzTn+ZNIBumx60biDUev6vtoRv4+12vUdrrfbLCTPSktEUgez7hm2kO+3JpkEi8KHRJBUQaJu4XDWfQofmLtNxZaCGHyyOQX5+kSsebIpvM9NL5lyGr+PKaVHZ5RM0hOJgWYTgJVPlTgkfWRL5BWasOSiFU/0W4iQrVOoQgMAixG2oGrw+I6rg9lkAxAVoMKiSS1xNCEH606n4I223vBb4J4kk+77ErlRvXAtx4K6UX5QyiSez1lZ4BtBo/JFw+ldZkfNPkCH1wVFjgABDwjK3SNn/Pjx+P777+Hj4+Nyu1arxbPPPotZs9zIJQX8J6CQSaCQitE5RgXxqaKZ9z7fk8TwDnWfPqUKYsdv1ErOLZ9+jQt6o1FciBoMBxYXecAE12CRcOBHdnLGriNRVJrzvlQBVGxBlYxIxDnvre9xYRv8Z+kvqDCTHjktny6djJLIWcAAlCFf38ljzbjkup3VQrPHmo9yLKt2f0fn6PxKYNJ2Ht+R30jOqAKBZhNZcJkNgA2MEbcbIldsCQz8jYaPbV+kWqc4opqxKBYg4A6RkFUUPV4Wj5wis8qyFImhajFu5N4LkZP419sJECCgJBRqBgT4RlK5qsvh2txsIj9bG17hdnI1jW+lRSpaL1+g4XAqV5NPcKwyuhnX8MUjHfvPS+Qa+cRujvpaLfS1K0hnStDEbcC2D6noazyW66A2gxf8VgvQ9iXH+qZJYVPjxOySBIRczREvqRew5jmHN11uApB5jWPPuQl8fd6hTEEqDSIRv8tsNsCgBUw3oFdXgFrxkIxXGQtJYsEGzOnD90XuDRz4iSrf7lNZI5QGkRtFvKc487qD6YHU/AmOtrtD47EkPuL30p+oQnPg6jYqux77A1hclIhrswJnFgEiMaTHfsdR9dtIydPjrV61ILMWQnJ5A9D1/ZL73/EJMHgm/7UrafwqwNbzS4jsJtZOsIbUhjWwKtYcv4VCoxhjdynwVqelqBdgglIqhl4eALnIDMX2D9w2/qytn4deEQT7ihfhr0RffyV61g2HKOWE57FFswEKYxaGzorDqmfaoH60v/vt7gXBVame06azNvUO4+f5r5qrAgQIKDeUO5Eze/ZsfP755yWIHJ1Ohzlz5ghEzn8U2QUGFBrNeK93LUQESrlgyFQs1A7+THf/VU+6LoRiKeWzJh0wt79DpWIsAPZ8CdzYwWjtiq0ot3UmRnwiAN+KQPUeXKScx5WcUWcgsP0T4JGPgCpdgbjdQNp53udfgZ1Fd2NfAUUqHlUQpb2j1zBJyx1q96Pax4708+wgFidyABoXV+0KtHqGqVarirpD3sE0I4zpAFRoyWMyaqneUYdwtj/jomuqVeIhmjj2+ppJG/1+JhmWm8BCtuHjQPuXBQmtgLtCfFYhFFIxAtQeRh5LQ0E61XVlSLsKU4mwJ8kKm81292lZ3qGly98FCBBQOrxD6DtXdyAbB1YLyRL72E2lNkxw8i/mtyb3BgK9qc6xIy/Z1bzfoOG+9n5FlYjclw0VYwHXN5EECK/PMaZ931BR4+XHURv/Cmy89J7GVMvcBCDlNP169n9PFQbA9LT2rwKnFtA7JekQvxdUIVzn7SNUkY3pmSdX05xXInOvzI1sTJVfcDWup2Y9rHoNZJL7rJz4O2E1AxBRlaPXAA1HsbFl1rPJlXyCqhafcIcCxRnqENYhxVHtERIvzvCNpD9OQSoVWJ3fBXZ+4qp4ajCMCmF9Hkm1fd+xDqzRCxi3kf5Dg2eRPEo4wPc2ph3y1RVh0ykwbWhDXErVIMegQGTTiRArA6nA2T/N8Ry3TgGb34FpyHyYLVZoCvUwyvzg6x8AlUgKWdpZ/n1J5DDWHgxju9fx1a5cSCVi5OlMuJ6hRaFXdbx5IAUbz6XAaktBl+r+mDZoIXxWjXE5H4Zag3DCuyt+mXcSL3atjuph3vD2Isknk4hh8mQbYH97xHJYbcBnGy5i+qim8FPeO0Go0ZmQnm/AvmsZsFqBtlWDEepbA/7hZVjPBQgQ8Lej3IgcjUYDm80Gm82G/Px8eHk5ZPcWiwUbNmxAaKhwwfhfRFaBAdfSC5CSW4hIfxVOZFnRpNHTCDCncyQp9QyLq+GLaA6ceZUFVN1BgG80PWLsJI4zjIWAUUfDwvg9HD8y6RgR2mAokHGB5nv9fuZtzvJvgIVhRAOmH4TUAFo8STIJYPFQmMPCdM0zrt4AMiX9bI78Bgz8nWTLzaMkXw7+7PocfhWYnLBohOO24BosUtwhpDrJmoM/chTLXuRoM0n+zO7D11KrD4mwym1ZAGdcBvq7MQ8sSKM/QeOxfM6Ob7CICqnJwkpWhvEYAf9pJGRpEeqrgLgs0eMF6bwAKwNC1SLoLUBGoQ2h6rt8bnUoCV2TXvibFyCgrJBI2MAw6Uh0tH+VqhyA64rqDkcmZUoSOylF/oqH/kfyZOVkV8N/iYy3K3xJuqx9znGfPo9EwND5DAoIr8/gAFUQEFSFozf1HgOaT+b2qae5/x5f8PhHLqVKT+YN+Eex4WI101/Prmyt0ZPPv+EVV8LByw/o+CZfr9yXj1MGwuITDbn0IfJsVAUA2fFU73afCuz9loma9nGpym2BOoM4sj3/MVeVjUQG9PoS2Pe96z5bPMW/ke5TgZPzWLtV61aUIHUc2PQalVKPfgtM3ktDa7OeDTmZikTI3H4OpYrVzNot6TAwcAYfa7NSSdzjMzax/GtioDkEk+exrhpVV8kxsfUvcExowK8cy9KmA9HNYW02AePXZOJ4Uj4WTmqJPJ0RKzbG43KqF6Y0n4UYXxt81SoYvQIxfMYppGmMmDGmKZ6ZfwJPtI/F3IMJ2HPV4cO0/UouJhl88eWwHQjWJ8Cqy0W2d3VsjLfiy8XxsFht2Hs1Ez+PaIQedSNuj96J1MF8He7Uor6RuKblaNjxhFwUGsz3TOTkaI2YuS8OP+285nL7mFaV8FyXagjy/pti7gUIEFBmlBuR4+/vD5FIBJFIhOrVq5e4XyQS4cMPPyyvwxHwgCAj34D3Vp/DxnOp2PJCO0yedwKP1A5DQKX2aGA+C6lpBTe8vh2I2wVU6cIiTHOTRsO9vnFvjBdej3P2c/twzKhWX6pq5GoWZhlXuECmXwBmdOUsdYPhwOlFnLev2Zvy31VPc3+X1nFcy148iKWAUUNCaex64NxKIPsao8RjO1LZknSYXf7O79LE8ZGPOFt+aS0TLGLas1hdPcVh2idXU3Gz5e2Sr0kkApo/CcwdAOhzXO/T59FUzyccOL+CP85oNpG+Qe6QcIBGlWuKfKbUIcCkHcIFrYAyISGrsGxGxwCJHHVQmR4apuIFUmK+FaHqu40gL2oiaJL5/SJAwD8Je+y1SHrXxt//GHR5TH7c9y2QEw9ENmGkfEDM3a8lqkCg09vA/MH8PfUsGzHDFzIRKquomVOtG9evkBrugwfMBvraBFdjQ8g+XhXdDOj5ObDpLR4vQPKlw2tcZ1NO0ajXTliIpUCX9znO9eejjtsvb+TjRq0mmZATT7Pc6t2oNLq+E0idBTR/EjYvP2ilgXiodA1SJRBYCcgqel/PFfP6i98HrH4KGDCDiphL63ieg2sA9QaTYKs/hGotrwA2mGBjHXXzGOs5iQywWvndu9rJf2b9S3z+ZhOBRiOBzW/ze9pqcj9upA5mVP3G1+hRWARbTAektfkaO+LTUD/aD3qjCaFxKyE+W5RctvsL1jx1+gMVmsFSeyCuaFXYez0eADD3UDyqhnhj1SmmTj2zxjGON7JFRbzeoyayCkzYfD4VBrMVzSoH4scdrkQIABxK0KDTjHxseK4tJs05juTcW7BYXf2V3ll1Do0qBiDSnwSN1C8S5sGzIZ3Tx9WzR6ZCao/f8ckmhmEEqGUQ3wffpSvp+SVIHACYfTABHWqEonNNodkuQMCDhnIjcnbu3AmbzYbOnTtj+fLlCAx0RNPK5XJUqlQJkZGR5XU4Ah4QHE/IwcZzqXi7V01YbUBcphbLjt9E88r1sctSF51r9Ib4wDRubLW4ph90fpeGgjJVSWO69q/RlM4+z3xxjUNx02gUXfeP/0mDxpNzihQ2KUBoDQAidnUCY0i81OnnKCC6vEdJt1kP+FcGlo0FOr3LIsQnkuZ7B39ydOeybwByFUe4jvzK/fb+nseVfBJYNs4hy/aN5Dx6XjJTOja/5Zjh9/IDHv2GBWzf79kBtHsGSeT0ALCYOEq27gWHOZ1IxK5jpbbAcqd4UGcEVweOOcmcq3alEiqqKQkxVRlipAX8ZxGXqUW9qDJefGrTeYFWBoSqWMgmaKxoGn6XD1YXFai5iQKRI+CfRW4icGYpcGEV144WT1GN8CCnHZl09G1b/6LjtvSL9CgZtZJNi7tFdFOqNrZ/xPU2bjfHgwcVpRDF7QGOzABGreA66yldcddnwMhlDAuwj0/ePApc2wGMWAoY87l26nPZyKkZSiWPMyIakCyyGJlWeXUL10izno85vxLo9B7Q6lmO91zdCmx4AWgyHshPAxYOBUavhr60dMgHFacX05g47QLTomDjmNWJP0nGJB8Hcm8AaRep7L22jedlyzsk2cLqcByu1TOA5hZrGpGE/kjadKByOzagtn9U8rnNOqqPxRLuM7xeSVWzHe1foyehzrXJJYrbjSjfn5FmGoludSLRPEgP2eIfXR+rzaBXIACLOgKzExvfvmv35UzUi/J3+5SLjyZhYOMoNI/xQkttIMa2qoxcneM99lFIUTvSFyaLDWeTcwEAaRoDErML3e4vp9CEbK3xNpEDANLIBjBP3gdc2w7bzePID6qLrIgOeH1bNq6lc0xrUrtYhNyjWqbQaMbve+I83j9993U0rRQA33tU/ZgsVqRrDCgwmKCQShDsI4f3w+IZJUDAA4hyI3I6dOgAAIiLi0PFihXv3sNAwL8OGp0JM/bewKvdqqFLrTDk6UyY/ngTFBrNOHQjC22qBiOl6nBEn55XMi7UvxIQWpO+MAN+BUQg0SOWsROmzykZbWnH2aVAy6dYcPT+jkTH1nfpowNQAl65LS/u6g0GdnxEg0SxlONcQ+cByycCuizOSovF7ER5Qt5NJkdpktmtmtOXhUP9ocCwBYBMTSm5MpDFiqSAF5OT91C5AxEL+nMrgN2fszDq+iELIYUfH3d9BzCzKy+Ce37Fot+oJcmVfZ0FsLv0CImcY1R2BZCXHztoC4ayaG37MuXoD0tXWMA/CrPFipRcHbrWKkPnzqRjkV/GvzWFVIQAhQiJZUmuUgcDEDkl2wgQ8A8gJwGY1c3VbyTxEBsBfX98cP3KCtKBTa+XvN1qJoEyfgvDB+4GygCg6QSqY/NTaa5bkE6C4NZpblOpDZB1rWgMuiH9TWr34zoNkAC4vpPntfkkrulmA49FJCWBowygQteoYQrWjmKEQvfP6MtSmAEYTWweSRXAkNnA8kn07zEbgNPzAFM+99X2BRI+fhWoJE47B1FBKmTqh2wdNWmB0wtYI1XpXBQxb6bCqvlTQMVTDI/Q3KIv4fnlrK1WvOzYR9p5EjdXtgCn5rPmMWqBat15/5nF/Pt29kQqjszLHB03FvD90iS73q8O5tpRjMSxQ5lyEK/1fwZ6XSbCZGKg8zvA4V/d+hCKs6+jSki727/7eEmhM1lKbAcAZqsNaRoDLFYbmscEwWyx4kRiLuQSMV7rUQMRfkocic+GQirGM52qYO/VTMgkpV/7lEg0E0sgDYqBzmcsTgT1wxcbL+HMJkckeOeaoejbMPKeFTkmiw1ZBQaP92cVGGGylDFMoAjZWiOWH7+JH7ZfRb7BDLEI6FY7HO/1qe1CXgkQIODOUe5mx+np6fj+++9x5QoVAzVq1MDw4cPRtGnT8j4UAf8wTBYrnulYBTqzFS8uOY3mlQIQ6qvApdR8tK4ajEpBKtzMNiFq9FqIDv5E0kYkZlFRtSuwbxoLph2fOKJEfSKomikticqsZwE3YjE7SgofkjhiCdD5PY5cZVzkheXaZx2Ps5pZdKRfpPIlv6gDaLORiPGULhDegNvO7uN6+5nF/Kk/hEkeW94BLq8HqnSiamj7R+w0ApSGd3mfBFP8PvruqEOA0asds+nVe7LrdeB7klQ2G19b88lA9e4sMM8tdzy/3JsePidmk8Cp2hVo9DglzPZZ9+vbgGbjBSJHwB0hJVcPs9WGMN8yjFZp7dHjZVeAhapFSNR4iAMuDRIZlWd5QgS5gH8IZgMvit2Zxl7ZxAvdB5XIyb7hPoUIoMJIl333RA5AwiCgEn8K0oG8Tbxgl3sDtfrRjP/6DuDkXBI4fb6nCmTlk/R+E4mZ8hhaE0g9BxgK2Oy4sJrq29wk7lfhDdQeAMDmShK0e5VqkaVj6LUCUJ3T+R3gxDyg6weOca3AWMA3imTR/MEOxWxUY6Dfz7AZCiAJ9r77c/BPo+tHQH4KjaGd66xOb5FIS9jP706Tlp5GhdlU+dpVwQpfnq+t7wHdP+W/VzayPpF7A62fpTF0YCyQcsL9MQTGUk2Vepa10eY3Xe9XBjren+Ko2BJo/SwCVo50HFNADNDlXaqpnFXeALICG6FuoKPeGdWqEladLEYcOUEuFWPmvnhEB6ggFQOXU/Px44hGmHcoAXudfHJ+23MDT7aPRaS/Er5eUmj05hL7ig5QItBDSIBSLkXDCgH4YXhjHLqRhXy9Ga2qBCHCz+u+eNd4yyXoUD0EJ5Ny3d7ftmoQfLzKfslosVix6mQyPt1w8fZtVhuw6XwqEnMKMXtcM4SUdSRbgID/MMqVyHnttdfw9ddfw9vbG7GxTCjYvXs3pk2bhldeeQVffPEXUY4C/lXwU8oQ5KNAXIYWP/WJhFoTB2XSLogjgmEK74akQgkqilIhmvEoUwke/a4oRSKAcZMdX2fUpLPyJv8WSY7Ra5h85U5urQ4h6WKzUt2yaypvf+Rj+tpse58JF3s8/D2mngFgBUJq8feLa+ivc2e83c0AAQAASURBVMxN4ppfNMmUq5tK3mdHeH1gy7tMyZB6Ma58wdCixIgiZF4Flk8Ahi1kh1bqRRPH6zvZqfXyY4G76Q0g7ZzjcYZ8YO/XgERKeXv9oZzj941kQaoOo2Q88QCQcJAGkGY9O2TNJlA2fegXJnaF1ipbMS7gP4N4e/R4WYgc+2fVy7/Mzx+qKqMiB+AFmKDIEfBPoTDLc+QyANvJuRBVal2mRLe/H391TPdwzNpMrtdiCdBgJJsSNhvP14wuriqM3V8AA38jWZN6hmv8xbVUwLZ7hefuykaqaH2j+Hk/tQCYuBVYOpakS0RDB2kW044KWmfcOk0lzqDfSQwpfLjONh3PcIWkg2wm6TXA2WUcA1v5BDBuEwrNYviXJVXvn4JYyvrgz0dL1llrnmUN0vYVEuD2wIdL64EOr1JBHNGQ53T7Rzw/Oz51kEEAFTa7PuP71PltYO0L9MPxq8DnOLWA71GdgVRF13yUyqDEg67hFJoUjm+VOH4JzYztdY0dOXHAiicYoBG/j6QfAPhE4KZXddzI0CJQLUf1MG80qRiAeYfcmA0DaFUlCCcTc5GnM2LRkUQMbhKNbRdT0alGqAuJY8f0PTfQs14EfhzRGOP/POrikaOQijFtaMNS1061Qgq1QorKwWqP25QVEokYA5tEYdaBOGh0riSTUibB2DaVIZeWPXEtLd+A77e7V11dSNEgJVcvEDkCBJQB5WafP3v2bPz444/44YcfkJWVhVOnTuHUqVPIzs7Gd999hx9++AFz5swpr8MR8A/DaLYgu8AAjc6Eur5aRG4Yi8BlA6A8/D0U29+F9+8tUd1yFaGniuLFfaM4L73lbWDGIyx4C7NYHLjD7i/Y6XGH9q+yKNnzJaAtMsXzCQeU/g5DYGVA6Rd1yceA5KNA1S70MqjUBqgzwLXIDq4O9PmBI1C6PPf7kchoUpx0iL/X7gecXuhK4thhNlCxM2wxMHQui6yzy4B5A5m+JZa6kjjOOPwrx6wWDgMOFSV4+FcEZAp2LVdPoezZrGc0aI1e3HbX55xJnzeABW12HIumh3HWX8DfjoTsQkjEIgSXpUNYkM7C28unzM9/b0ROsPt0EAECHgBYbCJkF3pQvfzTCIzxrIINqFw2nzVDAX1w5g4AptUF/tea0eI2G0d1lk8sOUpj1tMjrvUU19sTD1H9sXAYa4O931Ahe+sMTY8zLlOtcWE1mx1iKX30dn/p/tj0uSRo9HlscPT/HyCWcwQ6/SKw6U3g6AygVm+mYuo1QOJh2GyAwXxv4ynlCi9/hkl4GlM/9DPrnJPzHLeJREBhLlCxDf8//zESYz4RriSOy37+R8/BHp9zBGvj68CFNfT+G7OOqpmWTwNZN0iadXwDmLCVNV7blxzR46G1Xfdb9RESS+5STa1m1jx1BvDXCq2Q0HcpnlufjgC1HJ/0r4MedSPw9PwT+GVkY1QOUrk8vE6kLya3j8WcA/FoWjkQB29kISVPj/6NojDnYELJ5yvCvMMJaF45AJtfaIcJbSujXbVgPNu5Kja/0B4Nov09Pq48UCFAhRVPtUa7asG3b2sRE4gVT7dGhQBVKY/8axQazcjTea4bL6fme7xPgAABnlFuipyff/4ZU6dOxZQprgusTCbDc889B7PZjJ9++gmjR48ur0MS8A8hNU+PWfvjMLJFRRgMBoTHzYQk9ZTrRjYbJMnHgNST7MAp/am0sePWaXbQOr7BGfriHjW3TgM9v2CqxbWtLP68/On3YtYDq59ml8cnjJJq7zDg/CrnA2Bhak+TKg51KIuPrh8wznTNFJJKI5ayu6T0Z/JF/D6qW6p0pNlxcSgD2C20IzC2ZOKUM5JPAmH1imTsNqDV08CKSbwALU7iiKUOQkiXQz+csDqMY/eLdmzn5QdUaEE1EkAD6EXDXSPVARa6Oz7m65X7ULHjF+X5WAX855CQqUWoj6LknP+doCCNnwdR2fsLISoxMnRm6Ew2KGV3G0Ee4lnaL0DA3w1VEMeGj/zm9u6MqoOxcH88pnSuek+d8b8F3qGMi179jOvtEhlJDp+7dR8HR6QWPOb4XZcD7PwUSDrCdTf9gvvHFWZTseq8/gFUog5bACwb7wgRuLiGaUVZRUk9Jh3X9UEzGW5wdKbn40s9A7SaArR7rWgczsbjta+bhnwqUWr0YhpW/F5Iqz4GmeQhih836x11gTvcOgNYDBxfshtJ13yUtU5kI0fDxzuU598TKjSjmbV9TA3g+V0zhYqajMtseMV2IsmTehYIrgq0fZEEWvolxpYP/B04+jvNpm1WkmwJ+0s5/tPQDJiH9BqTsPumDT8vSoXOaIG/UobXl59B31q+WDg0ChHybKyc2AC5BXpYTTqYpGocTdHj+UWnIJeK0bhiAH7acQ3xmVo0rRRQKmGRVWCERCxG1VAfvNWrNgxmC7ykkvuSOnWvEIlEqBrqg59HNEauzgSbzQZ/pQx+qnvPWlNIJZBJRDBZ3I8+R/gLahwBAsqCcltRzp8/j379+nm8v3///jh//nx5HY6AfwgZ+QZMWXACv++l90u4RAP12bnuNy7MhM03mjPRe79xv83+74GGw0ve7hcN3NhN0+Ch82i22P9/JFa2vsdi7+gMJld5hwFB1Vw9bi6tdxgmFodMyf1nXWNRGFwLGLuBhJCkKLFi8zvcTuFDgqUg0/3+xFJ2LJ1eM3yjS253+3VFMXFj42s0GAyuBgxfzLEpZQCVS49+w5jWAb8CI5ey2JSpWNSMWgVE1KfywQ5VIIvw1s8ySj3/lntFEMDCt3JbdkbnDaI6R4CAIiRkFyLUp4zz+vlp9zRWBQBhRclVNwvKYngcwr9n60PUMRfw74FUwe9qn5Ljq4aqvXBYE4Q/9scjS/sAqnJkSqBWX2DSLiocIhoCTScCTx1g+uHdIj+Va5w7XNvqGIXxBKOWjYvix7jrC6o+AHrCDZ3HtbFiK+CxP6msvb6dqVURDUpvVPhFk3ReMBhQ+dO3pXjzA6C3X3g9jijBVjaS+5+CxVh6iqBfNAl4SdF3flRjfodnXQPqDqZ/EUDlUmmpa43HcKTdHfZPAxoMZWjD3q9Y/5j19DxaNoGqLamMRN/SMfS/GTqXZtNVH+GYlqeX51sBM07r0HV2Cj7efgu5hUZ8O6Q+vCVmbB8djle036DCnJaQ/tgQ/punoLLlBqrsfQmxB15Hl4BMPNkmEt8NbYiP1vLapXKQGiG+XmhdNdjjc/aqGw65lJdeErEIKrn0gSBxnOGrlKFioAqVgtT3hcQBgGAfOQY0cv95ClDJUCXk/vlH2Ww26E0WWK3uSSMBAv5NKDdFjkQigdHouQAxmUyQSB6wLpOA+46sAgMmtY/F0xIRDGYrZGKw6HKHc8th6/UNRDIvz9uYdO6Lp6bjGKmdcYmy3xFLqDIpjpNzgAbD2N2RyoH4vbz9wipgyByqUG4ec2wvU1Eqvf97/q4KBGr2oMw7J55kTXB1jlPt+YpKmYnb6XnT/mWqh47/SalybEeOUgEkYDTJjDHt+SWLSXeoPxRY9TT/f/BHoHIbYMnjjKgNrw/0+wnY8KqjwygS0RRy5FLAryIg95AMoA4BCnM4ulali/ttAHbY7NHqGRfpoeMb6Xl7Af8pxGdqyz6/X5B6T0bHgCOCPEljRbWAu1xPvEN54VKQJnhBCfhnEFAJqY+thdeVNfC/sRaQqZBWZwJO2arh1TVFo74P6rWJly8Q1YgNE5MOkKtJTpn0gNnI9fVOYchnU6TxaK4517a5Kldho4muQVPysSIxxySdyR6/CowZTz4GNB3LdCrvUPqk2LdT+ALdPi5S564AUi9ybGfFJPfP0WCEI6nLN5LpTJ6QchKoPxwq+UNW44pEVIkd+MF9ndXmeSD3Jt+H3t9xfG3di1T1+lWgV9D5Ffx7CKjMc243gXaGTOVQSRWH2cAfnyj3RuDHZrDek8hYJx6ezh+AxzN4pmvIg/Ou27wEv7RAdKllQ5UQb4xtHoFwaxpEBg1Eswe7jJSJLq2nSmzQTMjnD0LklbV4fMgyPL4lF/FZhQj39UKlYBX8lDI836UatpxPLTFGF+nnnuTJ0RqhM1kgEYkQXFZF6wMOpUyKlx6pjqTsQhy8kX379iC1HHPGN0eE370rckxmK5JzdVhxIhmnb+aiZrgPhjSrgGh/JRSyh+yzJ0DAHaLcFDmNGzfG/PnzPd4/d+5cNG7cuLwOR8A/gDydEQeuZ+KFRafw1LwTKNCbYJIomVjgDvmpKFRFwfZXXXqxEx8pEgFNxrHoyLhElcqIJcCer90/1mxgR0ebxc5TQAxvt5hIztQdzK5dj89J4EzY6khlePQbYNRKACKg9fMsIk7MYfrU3m9I4tR4lKqcVs+wCAmuzgSNWn1J+hRmA2eXAH1/IJmjzWTR1+F119clkTG16sYuR/FqtbC4lcg5Q273urGTOABJlwurgMtFBo/uYDFRnXRqHpB5jZ1ITwiu5poOcWaxZ/WOgP8UbDYbkrILEVZWw8L8otGqe4C/lwgyMZCYX4arXXUI/y0ebSvg4YSpkOR6+kV+Fz8kvl7XjQF48norTK/4Nb4N+QiP7QrE5JVJMFls6FIrDL7Kcg8bvTvIlGxwFGbRw23xCGDZOCpkCzL++vEAFaNNx/GzaMgHun3CH7uSND8NaPey+8c2GQdc2uD43S8a6DMN2P0Vf08+TgXrlndcyR6Dhv46DYZxNKswja+l5dOu3ncyJUevzq2g905kIypSShsJ9YkEpF7IKngA1VSlQSQFNKlUt8icmkAiEdMwrRYgvDbHs42FrDcG/kqvoPxkBiVENgJaPgMkn2JYQ/Exu0qt+fjSIJFT9eTuu9lm4+e7/asl7zProJUFw9j7R1eFlliCvLbv4o/r3lh/JhXPd6mGN3vWRKTuCsTb34fo7BL3vkC6HJKKVToDVgt8Nr+Ep5v6oEqIGvMntkCEH89RpUAVVj/T5rbXjEwiwuAm0Vg8uZVLzLbWYMbRuGyM/eMI2nyxA71+2Ivf99xAer4bT59/AcL9lPhpRGNsfL4dfhreCIufaIm1z7ZF7Ujf+2IAfvpmLrpP24MfdlzF7isZ+HXPDXT/bg+OxGfDfI/R6QIEPKgot4rglVdeQf/+/WEwGPDyyy8jLIwyy9TUVHzzzTeYNm0aVq5cWV6HI+AfwNW0Aqw+lYIQbzme7lQVqXl6VA0NQl7HT+C3sDdnmp1gCa2HE1kytIv2YsfL3RiPdxgLtf6/sOsTXp+dwMRDDhWKTMWUAk9IPQuE1+Wc99B5NMCzExS3TnEk6foObleQCkQ3A5pOYGrVrs/pzzN6LTBkLgmcW6fYeWoyjj48C4bSM6fD65QYH/+TBc2ySUD/n2g+fHkT4zzlanasIupTvZN9gwWi0p9qgcBYysDPr6RkW6bi7W1f4giXp4vQYzOBFpNpcFwc+anA4f/x/8YCnmdnzxxntHvFdczNy5/GkwL+80jPN0BvtiLMtwyjVcYC/tyjIkcsEpXd8Nge7ZybCESXYRxEwIMDzS36qZxZRAJH7g20eY7fyQ9qhHcRqoV5w2wFPt/lmriolEnwSrcaUCtk/9CR3QXykjl6m+GIGsaldVR3dP8M8A4p/bELh9EXxfmxtfsBXT6gT5ukaJSmzw803M24TAVIm+dhq9oVooxLXLe9Q9lgWfcSkFtkQFurHwmG4ohqDDQeSxJn0k6urafmAfWH8cI9/xbrDYUPsG8akHqaipQKzVkvVO3CseriEIlJVmjTkayxIUCtgP99Glf525GXyPfqxm7gid1sjulyWUfkp/B8LBntqM1GrwYWPU71U3hdplkd/xMYPwL4vRMwYjH9jSQKjpH7RQMZV/j++Vd0bzavDmaTbfFIx20SGd9vkw7IvwWrSAJrTBdIKrQE9k+DKDcRxtD6yGz4DKbuNUKEqvhw3H4g8yqMJhMKfKtg5kktFp7iZ2zqhouYOzQWsjXPArX6AFdKSRlN2E/V8rXtQE4cOlaUol7NFrDZbLiWXgBfLylCfb1QM8IXP49oDI3eBJFIhEC1DEqZ6yXX8YQcjJ515PbvWVojPt90CUfjs/HVY/URqL73WPEHDUHeCgR5K1Arwve+7jctT4/nFp4soYIyW214duFJbHiunQuJJkDAvwXlRuT07t0b3333HV555RV888038PMjA5+XlwepVIqvv/4avXv3Lq/DEVDOuJldiKScQnSoEYLudcKhNZhhtlgx/3ACnm1XBwUj18F7xzs0G5WpoG0wBsbGE1E1Jw0wq4CeXzF+2zl9QKqgofGqp1kwSGT0t+j2EbtDIhkJj4otgSZj6Y3jDrV6A+tfBvT5JEoKs2hanHmZiQ3zHysyFy5C0hEqcE4tcKRcze0HdHyTyhqxjKRU9g0mSmmSgRywOB22EGj9Ags/pQ+VNAA7x6ufcfjq6HJIJD11iP9eXs/kqcIsXpQ0GgkMmkWZvcXEgrU0o1aTzrN02WJwvW/7hzQNjGvABC1DPmf827zA48284ti2ydgHNA5XQHkjIYvd7XuKHi9Luk0xhChFSCoLkSNT8+JNiCB/uFGYzWjka04X1cYCYOdUjvh0eM1zwtIDgFAfL/w8sjHWnE7B7APxKDCY0blGKJ7pVBWViiXnPJCwWLg2OpM4dpxdWkSmeSByLBaOQjuTOHZcWA3U6gvD4LlQXFoHnJxLhWvj0byoL8wErm6DtVoPSOL2AueWcu1y9r7zjeJ3TPZ11323eZ737f6cpIRUAdQfDrR9ng2SJaNpRi2WAw2GA+2KGicX1nDMWuFLj53UsyXHf3p9DYikEGkzEa4MQm6h6eEhcjQpJMT0ucCfvek9YzaScLm2lUpfZ0Xute1AWG02gRS+QOUOQJd3SI75V6D6+eYxqqIU3vysWs1U+nb/DFgx0fX9ksiBgTNYS8m9qchq9zIQ1YRKO7k34F8RBv+qeGVrDp7p1BBHoj+CV7QRF7KsWDY/E4VGjoTtjS/A1AGN8M7a88jWuhJGmQVGiPQ5JKqqdC7dq00ZwO+TIojFYryz6hx2Xk6HzQZUCFTio7510SwmEL5KGXyV7onX9Hw93lvtPmV0+6V0pOYZ/pVETqHBjPR8Aw5cz0S+3ow2VYMR4eeFoLIkXTohq9CIlDz3SqbcQhMy8g0CkSPgX4ly1eg+++yzGDBgAJYuXYqrV68CAKpXr45BgwahQgXPhmQCHm7EZWox4vdDuJWnx6f96+K1ZWfwXJdqiPTzQo+6Eejyw2HMGNsMl6p+g/qtZTBYAD+VAjFbX0dA3Dag1zfslg1fzG5IxkUmGFRuC+z7zjWtSaYCvAIAVQgwr7/j9qHzgQotHTHfdjQYziJBl0NCQpfN2yxG3n5mSckXpA7mz5HpjtssJiZU7PmKxzmnr/uTsfVdKnd+78gEqeJjSSado5ARiSkbvrQe2DXVsY2xgKSONtNhJGkqBPwreX4TFD4lzR/tkHqxOLFHuZp0LFyrdeP4WGgdnuMdn7gqm1o+TeJLgAAACVn0sbonIuceR6sAIFglRmJ+GYgckYhpdAKR83CjIN2VxHHGoZ954R9QynflA4AwXy9MaBODfg0iYbUBvkopVPIHfKTKjsIM4MSfnu8/OhOo2MJ1dPj2Y9NLfazt8iasinod7WqMQeTFNWwqbHmHd0pkyOg7Hz/syMAHbUdDcmNnST+WR78FrmwFgmsAWUVkTnQzGkw7myubDTyOjAtAr2+5Dmszed+lNYBCzfXeDvtYVq+vgew4GvL6RgJVOtE7R8JQg/y0Avjch++4ckNITY6L2xVMBWnAykkOn7ziMOtZZ0Q2poqm5xc8H9qLbMatex7oPY0JpM7vTdZ1NsAm7eBY3K1TVDzXHQhABMv1HZB0+5hKu4trXGPhJXJ4DfgV/WvUwrDfDuGnEY0x5o8jJQ4xt9AEm02EbDdm4U0qBcBgsfGC6OLa2yljblHvMar9ACC0NrbGm7DjEl+LXCJGrXBfbLmQijBfBWpHeh4ZKzSYEZ/l2bT7eEI2akfeX9XKP40CvQlrz9zCWyvPurw/XWuF4rOB9RBS1rFsAJa/CCkwCyEGAv6lKPfKIDo6Gi+++GJ5P62AfwjZWiNeWHQSt/L08FVK4auUoWVsEPxVMlhtwMZzqSgwWrD5/C2EePui37zLaFzRH3/UPgl53DbuxGYB1j7POMoajwKtnmJ39cD3JQuKGr3YcVswxPX2FZPocdP2BRrfSb2Aao8wwtS+KFdsBWRfA7TpwOXNTMNqOBI4vcDxPAGVOS/uXEg4IyDGdSRJ4UOfnaAqRZ4BSzlXbjGxe9feQzIHQCJFJAL2f+f+/nPLOc/Pk8R9BsaSgCqOFpPZzbLZSipovCOANi8C25wUSzYr5cVp54Hxm2nMrA4GTi+mqWXDx3kxdB8UFAL+HUjMLkSgWn47keOukJ9Kub28jEbJTghViXAwxQqbzXb3c/fqYI4TCHh4Udr7Z9K5N8h9ACEWixBaFlL0H4TNZoPBaIaXpeTF8m2YdVyH9Bo2IGRqwMunaAdwVb8Wg8iiR4ifGjOv6DF0wEaEpe2Fb9ph2MLqIjW6B97ZmYsdV29BowvGtCFzIbp1iqoRLz8mH0kUwNpnqTi9spHH0WQMmxTukHQEMGn5vWAncrLj2IRxhkRG9c/SMUD1nkC3T4ED04DFo4oM1DNgazEZ+5L1GFbxIRiNs0MiZZy3HTePATEd6NXnDC9/vs5KbRgF3uurItJlFZCbxPGziHpAv+lsCg0uCqKI308lVN2BQNxeYOEQYPAsjpYr/DnSfnQmsoeth1gkRtCtMyRanGExQrR8ArpO3ofh9QOw63I62lcLwe4rJf2YFLKSa5OXTIwBjaKQasxHlfD6rDMN+SRszi513bjBMDa8NClUjvf6CVMX8++ib4NI9GsYic3n03AzR4e1p1OglEtRMVAFiVgErcGMNI0eOy6lI0drRJtqwfi0f118uuHibdWQM/yUD4lq6y6QkqvHmyvOlrh928V0tD17C2NaVS6zV06gWgFfpRQaXUnPRoVU/NB9lwoQcKd4SFo8Ah5WZGuNOH0zDwAQE6SGxWqDQirG0F8P4dshDTC+vgLvNYmAKO86RL4RmPhqLVj0BZBrKgHDF5Gs8K/IxRXgiFH1bq7pSXYofEjUzOxWMp7UVEjDxcn7inxo0jjKZB8pavMCIAKlv5c38rYLK4Hmk4BRq6mcMeuBgFiaBluM9NFoPpmkEMARpQtrHKqCGr04enRiDlOofCM5fiWRMRbz6mbGoTcZBxz/w/V4lQH0o8m+4So1Lo6CdJoHmo303xk8i2Niycd5v0RGP59qPZi4cGUzvQack6YkEqDhCECfAxz82VFIRzTk/uwRrD7hjGwVIMAN4rMKy+aPA9C8VBV4X8b0QlUiaE1ArsGGAK+7JXJCeOEh4OGFKqj0+2UPwXjSQwir1YbztzRYeigdb1TrA9XJme43bPMiDf13fU5FTXANoOPrVGCoAoBa/ZlE5AZZ1YZg+p4EVA/zRqooFN8kNMWHfccg3E8JUa4OMvl5iMXAkPp+rBnSztHYWCxjepYyCBjwG8dy+vzI5oXcx30akh0pJ+nFs3gkaw6rGUg4yAv9uD1Ub3iHsVFj9+zb9AaJIjtOzAaajket6GAEqB4iIqcwx9Uk/NQ8YMCv9CA064Gm44EaPXn+vPwZsd7vZyDxALBysuNxVzaxthgym/9PPAjIvABYAdioztr9GbfNiedzylTA1ndhbvcaftiXhrdbK7kPd7BZITq7GFOqNsf0xCjUivApQeS0iAlEmI8CzWMCcSSOqUmNKvjjvT61IRUDSVolVB2/RsTy/kzvbP8qMGwBkLAfNpka1uo9IMqOg/j4LBS0eBHaWkOx7JoIN3NS0b5aMJpVDsTEOcdul6V7r2Zi9sEELJncCrHBaqw7k4I3VjiUKD/vuo6G0X74flhDTJ57HPakbLVcgkdqh6FJRX9YrA9ZXP1fYNkJz2rXGXvj0KtuRJkJlzAfBT7qWxcvLD5V4r63etVC6D2ObgkQ8KBCIHIE/C3ILDAgKbvQpdOgNZhRKUiFFxafQoBKhj4VjZCuehIi53jvoKqQ9PyCZIQuB2j+hGtaAgBsepPmxukX2TExFgCxndgtyUsuSeI4w5gPNJsIJJ8EqnTlPDfAQsJi4Iw3wCKtTn+qZ0yFnJnPus5jyU8D2r7IMaNNb7CAA3gh+shHQHRLIKwuu0yLhjtiO3MTWQB1eB3o8AYlyNHNWPjVfBQ48jtHu6o9wjGw1U8Dj3xY+okOqko/n3PL6QV0cT0Lq/avsBiSyIBLG/kazi5lJ+3UAiZ5+UawACvM4jE2GQ80m0RFkkwJqIpGyAQIuAMkZGoRWubEqlQaet8HhKjYdU3U2BBwt4fjHcKLEAEPL3wi6ZmS54aQq9KV32sC7hhpGj1uZBRg/7UshPt5oV21YIT5esGrWJxvSp4OI38/BJ3JgvFjJ6Dy5VWOtdGOZhOBrKtc2+zISwKub6NKps4AoPUUNlGcHysSw9JkArL86kIiTsf8I4nYcC4V04Y2hN7EkYkIfyW+fqwBLNoc+B76AqKjxcigls/QD2/zWzwOnwiOXCsDuP57Sl+UenFUavxWNlzC61MxIpbRr+XsUip6mk0kkZN3E+j8Dv9/cjbJH1MhbDYrMvINMFmtkErKLTD23iB3Ij19wosSvCQ0NZar2RxaMNQRVKHwZfPnzBI2i+zj1/Zze3U70OVdYN/39DGs3p1109WtbFiJRPTjs1k53u0TgezqQzGuUAe5Kd81MbM48m/BJ3sxhjd7CzPOlYz9fqFrdbyw+BR61ovA+DYxAGyIClDCarUhIUuHt1adRbVgFb4dsgVBVxbB+/oOmDJvoKDZ80gThWDYzLOYP74jNgZXxrkUEwric9A8horkUa0qYcqCkyV6i4VGC15acgq/jWqK15eXVKKcupmH/dey0LFGKHZcSsfjLSuhXbVgbDp3Cy8vO41WsUHo3ygKFQJUEN8loXMrT4fErEIk5+oQE6xGVIDS4/qcrTUiXaPHicQceCtkaFjRH6E+ihKf8XuBzWZDSq7npmRWgREWTyN7dwCpRIwutUKx9MlW+HbLZVxNL0DlYDVe6FoddSN9hfhxAf9aCESOgPuOdI0eLy85jb3XMvG/xxtDKZNAZ7JAJhXjUqoGr3avgWG1vSDd9LwriQMwOnvzWyQiNrwKHPyJhIhY4iBETIXA0rGU6zYdz+L89AImUwyaUeQtY2VxpvChlN5qITkhEgMzugLD5lMps/GVonEjMdMKBs+ihDr9ArBsPGflAT5X35/5HBkXgMdX0FDTOWmrMBtY8xwwcTvnwFdMdByzM/Z8xUSqym2ANVOoClIHUzlTbyiQfIzmyTYbcOuM5wQp3yjH+Wr3KgunvcNKbgcAyUeBRo+TyEk9Q58hTTLjyv0iWZRmXadiqFo3h9RdgIA7REJ2IWqWNYkiP4VjifcBoSoWvEn5VjQIvcviTR1CZZuhgGacAh4++EYAI5cBc/u7XviF1WMMtfIvoo4F3EZKrg7j/zyKS6kOM3yJWITpIxujXfUQlwu9c8kaaPS8YB+3OgOzBqxD6KU5UF9bC0gVsDSZAEmNHsCv7d0/2fqXme4UUJleKYd/Ay6sAmI7wNJkInB5PaoffgO/VmqArI598eaOXKw7k4I3e9W6vQsfLxmgyeb4TfUebGYk7Kd6pOajNO/v9Baw5V2H31uLp4Da/YFzy0oek9yba/O6F7iWt5gMbPsA2PQ67xdLgfpDgHEbWa/sKYo4l8g5stXnB2Dtc4BfBWSZ5IgN8S6RXPRAQ+7NMTKTDuj5JYMQ7KERcjWJmo5vcNQdYK21aDgwahUfs+19jmfb99V6CsfbvMPoM2S18v+LH+c2nd8DrDbg+g6YWzwNc6Ox8L51FqErx1AhE9WY413uENkIODoTQYZEPFKrCSL9lbiVp0PL2EBIxWKkaXR4vks1nE3WIC6zAG2rhuBWrg7Td1/H+DYx0OjMOJ6kQec/8tG5eh/UjxiALIMIx9bk4eVuYYgKUOJQggY/7aefm0gEPNOpKvxVicjTmUqkJdlxJa0A6fkGj6d4xYmbWDipJTrXDIXeZMHkucdv33foRjZ+3xuHpU+2uquUp2vp+Xh8xhGkahzmv9VCvfHHuGaIDnBVJGbkG/D+mnPYcNahSpOKRZg2rCE61wy9b95cIpEIj9QOw7oz7sm4ZjGBUN/jc/l4ydCsciCmj2oKndECL5n44TEWFyCgjHiIVhQBDwOsVhtWn0rB3mucG151MgVjW1fG/3ZfR6MKAagUqMaCw5cwqWYARNe2ud9JxmWSFHZC5sIaoNEoxlg6I+kIEFAFqNaTapQKLdhda/4kEFyFBYI2k4lOOfFFhEkY0Olt4PB016hQm5WpGBYTx4yO/FbyuRYOZVqFUUeCyeZm4bZZKf9tNoHP6Q7B1ZmItekNx23aTGD3FywAz690jI0d+ZWpDetfct2fKpAXJZve4IXnsRmUpXtC+kUgtEh9JBKxmLq6Gej8NlVIZj3Q/mXAkEeSp0Jzz/sSIKAY8nQm5OlMCC/LaJXVwhHB6Gb35VjUMkAlJZFz9w8uStPRJHNMQMDDidCawKTtQE4CR+WCq3FN8Qn7p4/soYHOaMb32664kDgAYLHa8NT8E9j+cgdUCnJ4WsVlFjj9X4tH/ohHz9oD8EizITBagDa1aiKi8KJL4o8LDBqgIIMK1YDKQNcPYGvzPEy3zkH+Z/fbIz6+2Azfoz/g+wFL8fSeAhhMxT7nxgKgy3scW5apgGbjAV0e18xafYElo1zHlU/8CQyZwzEv+wg3QKKi/y/A/mn83b8SsPYF13RIq5kKV7GMx337JBmpsG3xJFCzD2y1++G9nTl4tVto6Sf9QYNZDzzyMdXKq54CtE7jSkYtiavuUzmGfesUb7eYeD5WjnZVVRkLOE4XEANUaEqfmfB6TB9rNBqoOwg2L38UiLyxwzcEEnUQ6uVpUWnVOK4RJ+Yw2WrpmJLH6RNBFV7WNVjMFjyz9ASC1HI0rRyAmuG+2HEpHV9tvgyZREx1ip8XovyVkEnFuJyaD73Z0XCzWG04cCMXravVQFU/L4QEaGG02PD9sIZ4e6VDVWOzAQuOJOK9PrWhd+Nx4wyTxfNapNGb4a+Wo121YHT+ZneJ+wsMZry18ixmjmmGQPVfkxJpGj3G/XnUhcQBgKvpBXht2Rn8MrLxbXLDZrNh/dkUFxIHcER2b3uxA6qE3r+GRrPKgYj08yqRLiURi/B69xoeE77uFn5KGfzu074ECHjQ8Y8QObm5uVi2bBmuX7+OV199FYGBgThx4gTCwsIQFRX1TxySgPuEjAIDZu5zJBula/R4plMVRPp7ITbEG2P+OAKTxQaR8S8WJH0uFTRGLXDgB2DyHs6yH/+DBYFcXTSf3Qv4X3Mu9CIR5c8Nh1MN4xxhGtkYGDAdMBdyZn7ru+6f98pGoOk49/dl36DMWh3G4sPjSXATu+qMxqOBXZ+5v8+k4ziUHYXZlKF3/QCAiGROWD2SU2umOJI3pF4kvkqD3Zi4Upuiojnd0QkD2K2s0IKjX7rc+zbqIuDfj8Si9I0yzbcXZvCCSPkX3iZ3CJFIhFCVCDfLROQUXWjlJQlEzsMO3yj+PNgBVQ8ssrVGrDyZ4vY+s9WGI3HZLkRO3Si/EtusPZeJteeAUB8FVtQHx3JKg9hpDZPKAasJ8lUTXX1aAMCsR9iWp/Fih4WutlqaFKZPJTtUDdDnAA1HsXkRv7ek55xJByyfSD8UvYZEjXcYlV37plEBIpYyPtuZxHHG6YVU855e6Hr7iTnA2PWwKYMQciMfUrEI2QUGBD4sfh0WE7DnS6DpRFcSxxmH/kdvwvWn+HtgLBXNxcfq7Nj1OdD3RxpQ5yWRPE89DcwfBJHZAEX9UTAETYBBb0O46JzjvdekAJc3kFzb9QWQm8CaL6Yj0PpZqqbEUqTIKiC3MBEysRgjmlfC8N8PIcLPCzPGNMVPO65hUONoyCQivLXyHL56rD4KTRYEqhUQiygGUkjF+H54I/y84xpOJuXePmxfLym+GdIAWuNVnEumYfrWC2mwWW14+9FaUMnFKDSWXHMi/LxKJWCaVQ6AWi7B/ms5sFjdjxadTMxFns54R0ROusaApGz3I0wHrmchW2u8TeSk5xvw62434RggUbX2dApeeKT6Xz7nnSLSX4lFk1vii42Xsel8KixWG+pE+uKjfnXuK2EkQMB/CeVO5Jw5cwZdu3aFn58f4uPjMWnSJAQGBmLFihVITEzEnDlzyvuQBNxHWG025Opoltu0UgDGtqmM4b8dwjOdqmDB4QSYLFyorHKf0ufSVcEOrxu/aMBipunxkDkkcQqzeNE1q5tDGSOW00Bv5WRXEgdgAbbhVaDbJyUjSZ1hs3F22xNy4lmoBFVx7d45I6gaR7pCa7OgKQ7faAcBUxxXNzMpK+Ukf1f4ksxa+STVRv3+Rwm4byS7WRmXuJ3VxAtPu4pJWpQApMvl79FNHSafITX5b3GFE8ARrptH6QMgQMAdIv5eosc1RVLr+5iAFqwUI0lTBiJHFcTPkGB4LOA/DrPVBmMpSoLMAtdxkWqh3ogOUOJmDi8iRSJ2xvUmCya0jcGMvTfwSssAeKsC2aAoDnVICf8iUUGG+20BIC8JrULNsHkVXfxaLVTHJB9nE0jqxeZPu1eBvV9z1Mq+XhaHPg9Y9yLHhy5vZCMp7yZrFFUg/9XlejwXsBjdK3RNhYBBA5F3GIY3r4iMAj1k0ofIbNuQT7+/m0f4hnr5c9zc2YfQTsbY4V+JCmBPyInj4xcMpTq5wTBHwAQA+Zm5aNe/L1L8K8FysxgZdGYxR7XavgAoA6ncOrccWD4B0OXA0OUTXNAo8PnAeghUy/HswpNIydUjJVeP9WduoV/DSET5e2HY7xxVPxafg/bVQrD5fCpGtqiEuYcS0LdhJNacSrlN4rSKDUKjiv4wWWz4fttVvNq9Bsb8cRShPgp82yscTfwKIL+1CYdGR+NQlhJvbU1HZgFrYJEImDqgHkK9FagR7o3Lqa5qNLEIeOfR2vBXyW/Xxp7gLj270GBGptYIo9kCtUKKMB+v2/W3Jzj7VlqtNmSUMvaVlFNKLVxGVAxU48vB9fFGz5qw2mzwVkgR9LAQmwIEPIAodyLnpZdewtixY/Hll1/Cx8fhw9GrVy+MGDGivA9HwH2GGCI0rxyIPVcz8WTHKpiy4AT0JiuUcimOxufc3m5XsgjdGoyA6KQb4q5yO+DWacd4UctnOAoV2YCeB+tfAqp0IbHhXDzZzCSG7DPZxRG3myMTzkWHOxQ3V3aGbxS7Qf1+4vx+cXM2kYgxp7dOA31/AGb3LWm+7B1CcirPjYN/6lkgqgnQ+jl66GgzuU9lIJMygqqQyJHIgKHzHPGY2iz6evT8kgSOTEWzaN8IIO0CiRm7Yqdia+DcCs+v8cxikkkCBNwhErML4eMlhbeiDEtK/i124pWljAbeJUJUIlzKLgORI5bwYtLdZ1OAgP8QVHIpqoR443qG+1Go5jGuCrpwPyXmT2yB15efRpsoOfpUlUGZnwCVjz8KlFZMWpWFUUk5WNzvN8gXD3X1jxNLaXbsE+H6JNbSL0olmiSg4BZQpRPJmMRDjLa2WdgAiWrCte78CiY7BtdgcqM7yL259vtXAJJucS31i2LjxzeKozueSCiRmK+hxAHKAYUfYDUhUpSDBInvfRsfKR+I6B9k1jOEQZPC+kIkAQ78yAaZOoRKJjuM2pLx7M7wCXckY9rHsIoh7OKfSG1RD5IKTUo+Pu0cSbfAWNi6T4XoyiZYI5tC1+I5FAbUQLhGCoPZitM3c/HiI9WgkEogEYuQrtEjxFsBL7nktvpm6bEk/DSiMV5cfAqTO1TB+31qo1KgCpPnHUeUvxKf9K+LI3HZ2H0lA0qZBI81rYBAbwXWPNMa1eRZUC5+jEptAL4AugVXR92RczBmZToqBKrwYtfqqBKqRp7OhE/718PiY0lYezoFepMVjSr4453etVEjnNdBDaI9e3dVC/WGX7G0s5RcHb7YdAnrz9yC2WpDiLcCr/WogWaVPTdEFFKxy8iRUi5Bo4r+LrW5MzrU+HtGAdUKKdRlqRUECBBQAuX+STp69Ch+/fXXErdHRUUhNbWUCEgBDwVsIhvGt41BSq4eaRo9utcJR9NKAagf7Ydgb8Vt07fnV8fh8NMvwlcsgejUfC7sIjFQ41Gg0Qhg2QSqSlo+Q++MKp2B1HNA8hEqc2TKkrP2IknpXTOA3aTk454NhCu1dRQZxeEdxiIm8zL30f9XYMMrHFMCqJ7p+j69dk4v4nOMXg1c2kCz4YDKjBq/to3pUNveL/kcMiULQu9wjj3ZZcVSBefD086R6Irfy+JJIufx6nNJEElk7E5pnCTx1R4BYjqQdBq5jKaAZ5eUfG6xlDHsUU2obhIg4A6RkKUte/S4JoVEpfNYxT0iRCXC9gQrrDYbxHcbaa4WiBwBAkJ8FPigb22MmnmkxH2NK/qjYmDJhkelIDXmDo2FZMeHEC9ccPt2H2UAZvaZi6d3Ah+eCcCHkw9AeuJPIO0smwxNxnFdL/4d4B1OZY3Z1VMDANdbuTcTr0JrM2Gp0ePA2ue5HgJsgtQfyvCB9S8BI1cAh35xrwRu9Qyw6mmgzbP00Vs52bUpFFYXeGw205R0xS58a/YGru8ouc+6gwBVAESF2fDNiUelyLYP1wWsTygbQdd30EvI3rjy8qdH35HfgZh2DJsAeI66vMvkOLmapE5xNJvE9w0AqncDEvaV2ERs0ABWM6xKX47Cx+0puZ/O70GkSYGt3SswhdRFnM4bg39g41AmEaF/oyjUivDFlAUnYDBbERusxheD6iPC1wuP1A7D5vNp0OjNeHf1OXwxuD7Op2iQkKVHowr+AIDPBtbDq8tOI03jUKwcS8hBpxoh+KJHJJQrxtwmcW4j8woitz2D1eMXwuoVBG8vKdLz9Rj3x1FczyhA9zrh+GpwA0jEIlxNL0BmgeG2YXiwtwJPdojF9GKjTlKxCJ8NrAd/pQzJuTpoDWYopGI8Nf8ELqQ4CLSMAgNeXXYGXz9WH5PbxeDXvXEojnFtKiPEx7FO+6vkeLNnLQyafqBETzLc1wtNK92/5ooAAQL+HpR7BqJCoYBGoylx+5UrVxAS8hdKCQEPPBQSCRKytPh1dBOE+CgwulVlbL+UjleWncFjTR1mgHqTFc1/voQtkU/COGkvbBO20gen45uAdyQw4Fd21rQZlO7G7QGO/soUp7YvMymhVl/HE4fUBLp/ytEITxBLOHe/cyoLu6hi3Z6opkCnN6nyqdnb9T7/SsDA3xzFnE8kEFoLmLAFGLseGLeJSVbnV5HEAUgUzerG+e+mE4BHPmEBs/sLEjZNJ7j62qiDue/8FGDLW66+AGYDC1G5mtHkAMkdm4UETMunSL4sGe1K4gA0dT45l9uH1gEKM4E6A123qTMQmLyXBfPer4F1zwM3djvGXgQIKAXxmYUIKXP0eMp9HasCSOQYrUBmYRniTNUhQG7ifT0eAQIeRjSqGIAFE1ugRhhVAyq5BBPbxeCXkY3df96tFsjOLoL49ALX23U5CFs5BJ929sf842mIQxTQ7RMYH1uAW83fxrRTNry59iquJ96EKf0KFa058SQMun5Q8nkUvsCYNRzTST4BrH4KyE8FVj7hIHEAEg+nF1FVE9UEOPSzw5/FDomMaUlZV4Hs60BhDsmg4sretHPA9o9I8jgjtjOPMb4YIVG9Oz388tMAuRoiQz5Uplw8VJAoqGI+/oer+lifS6Kry3tA9Z40dR42nwle2XFURw38zVX9LBIDjcfQe8+goTKnWjfX0IkiaKv1xYebEyCWyIEeXwDtXuHfAsAabNBM4NoWYP1LEK18AorfWqPitfkY05j1n8liw6qTydhzJR1v9qoJsQi4kanFmD+OoNBowTuP1r5N2CRkFWLi7GM4dzMPI5pXRIHBgkGNo7D+7C0XEseOnZcz4GPN4d+DOyQfh8qUA28v6e39X0rNh8liw7ozt/DswpN4ev4JfLf1Cj5Ycx6pRca/PkoZJrePxayxTdG4YgCiA5To1zASG55rh8rBKvyxPx49p+3BwF8O4FxynguJ44wvN13GiJaVMLJFRciLYu7Vcgle6FoNE9vGlogUrxnug7njWyAmmH5XIhHQpVYoFk9uiUj/UtTpAgQIeCBQ7q2Bvn374qOPPsKSJVQEiEQiJCYm4vXXX8egQYPK+3AE3GcEqOVoXz0EUxacxPi2MZi6/iLii4xQYwIU2DGxCiz6fBhFCiy6aMBzK+MwY3QTtLLEQXp6HsmGat2B9PPA/u85PrTzU9eZ64T9QMVWQK+v6QsTXg+o+ghN+bQZQNWuVL0UR93BLBrsCpruU1mQ6HJYcMhUwMJh7CK1mgKMXEovHoUvPW+ST1L9supJYORKFodXNwEX1wGwAWPWMw7cGTYb0xx6fslUKWMBULMPDRm7vAeM38yujkwFmHUkqpIOeT7BpxZQYaMMYPpD//+xwLm6lUohT4aEpxeQ7JnRhXPvj/3JMa2s6yTBWj9L0snglFByZSPQ7mUWad4PWdqGgHJFQrYWrWKD/3pDd8hLJlF6HxGqYgGblG9FqPou+xXqEHpCCBDwH4e3QorWVYMxf1IL6IwWSMQiBHvLIZd6MC0uSAMOfO/+PlMhgrNPoEpIJUjEIugswOYrBXhxySn4KWVYNiwalbc/BUlCkfpCIqN6o9UUqnX2fsO1MqQG1/5Vz7BJAlDVc2UTGx7ucGwmm0R2tU7vaVzTbTYAVkDuC/zZk9uG1mLMtjvcPEpl7PCFJCuUgSQkFo8Cmk/i6JY2nftOPEx17LhNjFKv+Shs7hQqDzLyU1mHuYPZwFrl3EpHYpWXH02jjQXAjk+Bbh+zeWXUkqy/vJEq5jHrGFF+fHZJbyG/CtBX7IBXVGZIj/1KJVCtvsColay/cuKAHR+XSAX1PfgFRox4FDOOApPaxaJZ5UCcuZkHqw2YOaYZtl1Mw/zDiVhxMhmT2sZgfNsYeMnESMnVI8hbjkKjBZPnHUekvxKvda+BiXPcx5yLRIC+QINSKQ6n9/m0k2FycdzK00NncowYBqgV6FwzDI0rBsBotsLbSwqZRIw/9sdh6gb6O9UI88GVNA/Jb6CBscVqw7u9a+PJDlWgM1mgkksQ5quATFLyc6tSSNG2WjCWTG6JfL0ZErEIgWo5fLwephFAAQL+uyh3Iuebb77B4MGDERoaCp1Ohw4dOiA1NRWtWrXCp59+Wt6HI+A+I02jw9JjN9llsOE2ifNjvwpokjwXPsd+IpEikeH9Lh/go449YEvYBFHKcaBSa5IwB34EavQAGj5Ozxh3xnmJBzmfPXQ+u2gLhvL2/T8Ag2eS3LiwknP4Ehnl1bEd2UWq0RM4vRio1pWPkymBBsNZGOjzivYzjT8yJQuWoKrAkLnAbx2A9q8Cx2cCF9dShVOlC2DQ0khx1ErgwE+O567VB2j7Igkeow44NQ9oPYWEy5HfedG44VWaFVstQI/PqUDyhNxEIKQW0H8658TTzlFFE1SF8/yeYDbQb8dO1Gx4hfu4tB6o+Siw+U1XEseOvd8A9QYLRI4Aj9CbLEjTGBDuV8bo8fzUkuq4e0SIiuNUN/NtaBJ+lw9Wh1DVZrVQxSdAwH8cwXdqRmoxeTYnBqDIu4ZWVRojUC1HRr4BLy89DZsN+OyRMFTdNsE1HMBi4hhU7f7Atg+A2v2ARqM4FnV9h4PEAThWXNq6mZ/q8OBKOsKfSTuAuQNJ7Aye5SB2pH+hQtAkA8vGMtSgw2skidLPA+tfplqkzgASDQDHwmUq4NgMoPHjsMk8+6A8kLBZ6GHmCZnXAXWRCjqsDtD1Q9ZWLSYDBakMaRBL6INjH4+LaMDv1zn9gLEbGC5xdhG3qfcYbM0mQCZWoln815Ccns3HSKSMOm84gvv0AO+UA/hyUG8cvJGNCbNdiZiJ7WLwdMcqOJesgdZoQaiPAo/PPAx/pRz5BhNCfBTwU8pw6ZYG/qX4GNlsgEnhT0an+DwSQOWRk99baaoWhVQMmaTk6K89VQoAknMK8dMOR4MwT2dyGY9yv08xvGQSVAi8c2PtEB8vhPj89XYCBAh4sFDuRI6fnx+2bt2Kffv24cyZMygoKEDjxo3RtWvXMu/z559/xldffYXU1FQ0aNAAP/74I5o3b+522xUrVmDq1Km4du0aTCYTqlWrhpdffhmjRo0q8/MLIDQ6EzILjFh67CaeaB+LhGySOKOahqOXbh0k+78gwdLyKaBKZ0j9KwGHfqZHjr2TtuNjYMRi4MQ8zs1veMnxBApfzpz7RbOYurSR41C7v3BsY9YDy8azoGg1hRGVYgm7dSsn07Cw41skh9TBQMWWNEiEyEPqRFGMo80KZF4BHvsDuLKFJsPtXiHZc3ULVStp54CUU0Dnd4DmE/m4lJPAH71ooFijF3DkV5I57V8DKjQDZGoeh3cY1S9iCbuLxVO37AirAxz9nUkP+hxHcZSfxoh1T/DydzWT1WYCC4fymPyii86BB8TtpQeBAAFukFj0OS9TYpU2jRdmqjKqeTxAKRXBR46yRZB7h/KYCtKYDidAgIA7g1RBdV1ugtu79aFNMKFSDPxVcuy4lA6L1QaFVIz6vgXuEx4lMl70Z17hhTwAtH0JuLHLdbvsOKpyPSG4uuu4ZO3+bNrYx7DOLAGaP8F0RxFKuUgXMbXysTlU8u77znUEy2bhD8A1veMbPHa/aNjEUhgUwXio8nmkCiqgPNUjlduSyGk6nmqkgnQqjze+DvT+nqNuJp3D2FodTCXy6mf4HbtrKtD+FaDFEyR+Ts6H6PdO8B06Hzg7z/E8InHRuS19VNZbZkXDMAl+2FGSTJyxNw4/DW8Ek8WK9Hw9aoT7YEqnqth/PQuT2sUiPV+PvEITGlbwh0IqRr+GkZh9wP3fcbbND341B0Bx0U1oRMPHmahahHpRflDKJC7KGzuGNK1QKikDAAUGCzR6h6dTqkaPEB8F1HIJtMaS+xzYKOov9ylAgIB/D/4x17W2bduibdu297yfxYsX46WXXsL06dPRokULTJs2Dd27d8fly5cRGlpSRRAYGIi3334bNWvWhFwux7p16zBu3DiEhoaie/fu93w8/1XkFhqRqtFDIhJhSueqyMjXw18lR5Bajrc7BELy248kGjq/DRz6H3D4VxZptfoCwxYCa6awYDPpgKXjaMp7bZuDXKk/lB25k/No9BtcncoWmdIR1W2H2QBseoNxo21f4piEWA48+i0Nh7OuAYEx9N3p8Tm7dTkJlFQXLxDtqDeE6VMrnneQJxVb0E8GKCr8rJQUb3qj5ONtUdwGoOx22/skVh5fyTGtnBvA8vHcx5C5wPmVJQ0ZJXJ2+xYO5Tno8AbPG8DHqUP4GtwpmJpP4vn2i6b6yK8CybDTCxntXhrcEVwCBBQhPvNeoseL/JzU95fIAYBQpRhJZSFy7L4OeTcFIkeAgLuBTzgN/5eNd3NfBHxjmiA4kF4c+UUXp0FqOaR57i+Y2WApdrEqctN0yb7BtU0d4n68uNM79MBp9QwQ3ZwqE2fD4iubgCZj2ZSRq+mRd3Ftyf3U6suQgSUeGn/1hvCYe31Fo+b1LwNVugLVe8Cm8EWhTQZfD6/0gYQykJ6EK58oeZ86BAivC/yvNVU2A34FDvzAcwkwRfTxFQyGyLzK2sQvmgrk/KJgk5tHmas9o4urobUz+QOw0dTxTSqfopsCN92PPSnCaqDK5rFY1mESDpmr4/m1N2F14n7WnrmFKZ2qwGSxwV8lx7jWldG4UgAmzj4Gg9nxN1U/2g/fDWmIzefSkKpxNdruUD0Eh26ZkFfrFTRWBkJ2Zi5rTqkXbE3GQ9T2BUDhfXv7CD8vzBnfHGP/OOJCvDSvHIgnOsRC4WlM0f4WyMSQiEWwOL2QX3Zew9dDGuDVpWdQYHDUb00rBeD5rtVK+OAIECDg34tyJ3J++OEHt7eLRCJ4eXmhatWqaN++PSRuZjnd4dtvv8WkSZMwbtw4AMD06dOxfv16zJo1C2+8UfKCumPHji6/P//885g9ezb27dsnEDllRG6hEStPJKNulB98vKQQiYCfdl7HzDFN8dXg+jBrk1mMdXmXSUz2+WGzgcqWxIP0kFn8OG8vzOI22XH0tbmxk8qZRU7x9FnXgMsbgH6/ADV6A5fcFF1XNjGhKaY9u0UBlYGDPwGX1lH6G1KTBsJZ1zk61ecHjngkH3fdT0AMUHcgsPtLR7EhV1PVYkfcHhIkCQfcn6QavYDrO11v0+Ww+5cXT98d+3k58huLou0fOrqIATE0VNz/PSXneTdpGigS0eC5+1Sex26fcLTrRtFzyVRAs4ncxqwHurxPVVDmVY5jdXyTqSBRjWka6Q4x7d3fLkAAqMhRSMWlytE9Iu8mCUav+395E6QS3RuRk5sIVHCv7BQgQIAHxHZi02T7R7cVL7bo5hANmA5FUIXbm9ljknN1Jph9PBCmFiObNc7JVTd2ATV7OXxZ7Nj6HjBgOr1ZUorWMmUAla4pJ4DkY0CH1znGLBJROSRVONTAEjkbG1YzzXRlSkaXW80MFKg/hEoeYwEQ3sB1tAvgelq9O42ad39JLxcA8AmFrUYvFEgCIHUON3gYYNbzdXb7hGPWdvLL3pTLvMrI+K4f8jy1e9VB5CQeAvZ8DVRuw7G3+L0lU57UofTVkchciRxpMUVJ3YFcI5KPUwm9dGzJJLOGI6geTjqMkKTD6N1wBLpMeRYZeVpc18rx2d5cZBUYUKA3o1KRsW+e3oxJc1xJHAA4czMP8w4lYM6E5lh/5ha2X0qDSi5Fn/qRkIhFeH/NOVhtwNonX4Ot6lgorDoYxUqER1VCoI/rWiaViFE1VI3Z45vjSloBsrVGVA31xs2cQry5/Ay+GdLwdhMkXaNHttYIg9mKQLUcIT4KBHkr0LNuONadcYy4nb6Zh+m7ruO7oQ0AiJBVYECdSF9E+CvvfAxSgAAB/wqUO5Hz3XffISMjA4WFhQgI4KhHTk4OVCoVvL29kZ6ejtjYWOzcuRMVKlQodV9GoxHHjx/Hm2++efs2sViMrl274uDBg395LDabDTt27MDly5fxxRdfuN3GYDDAYHAY6LlL3PqvI12jR4UgFSRiQCYRYfb+eLzRsyYi/ZWw2mxQir1IyJxe4j6OMu8miZnwevTEAVgsBcTyQiq0NqW47rDpDZoSuyNyRGJGgOcm0T9mVjcWIjV68r4lox3bJh8HZnWnAV/aOeDEHBZ8tfrS52bnZ0y0imxEMqggzVVFkHae3fuwuiXTDIKqAJENHdJwO5QBVCQYta7dwatbeE6GzOXFpEjE59v5KWXaITWpRvIOA8ZuZOzqlU18/qXjgKbjqMCxFBWsZ5fSv0emZEHV/jV2vGRKvm5jIdD3Z2BGZ8comR3NnwCUpSSBlSOEz+KDifgsLcJ8vSC625hvgH/n6hDX9Lb7hBClCGczS0rP/xJyNQ06hQhyjxA+iw85jFqOwRRmcR1QhTBq+n5AFch0omrdSORIvSBSBZVIpgv3U6Bb7TBsuZCG68ZARPlXdJsWZ82Jh6HrZ1BuepFjx80n8d+zy7ge2pF9A9j7HdUwBalUmlpNNNSN38ttaj7KRocmGabH5kHS40uINUlAxbYcqZywlefFagHqDOKPWU+PFmOhY5Rq/GamVJ1ewG1r9AQi6gNz+gL1h7HxlBPHUela/SC6sgnqOv2RbwsB7vNw1d/6WTQbSWY9NoevWZ9HUs1qAVZNBlo8TXPjzW+xdpuwBYjpAMTt5uPj97BptuNj92NRjUcBh6ezkbfqKcftKSfpmZhwgCRaZGPWNn1/5N/syGXAqflMBlWHcFxfkwxseef2LkSnFkBVqw8qrRiHSupg1On2M3YVhCLS3wvhRcTJ2Zu50JtI4iikYrzYNhTdY6SQmzQwy4zwEWvRtXYoZBIRcgpNmLHvBhKKfB+HN6+ANeezMX2342/2lxFB6FW/ZFMiPqsQg6cfRIi3Aj5eUqTk6W4/7/qztzC2VSVcSS/AU/NOIK5I4SqXiPFUxyoY3boS3upVCzdzCnEqKe/2Pm/m6BDlr0StCN+yrb0CBAj4V6DciZypU6fit99+w4wZM1ClShUAwLVr1zB58mQ88cQTaNOmDYYNG4YXX3wRy5YtK3VfmZmZsFgsCAsLc7k9LCwMly5d8vi4vLw8REVFwWAwQCKR4JdffsEjj7ifr/7ss8/w4YceEgwEAAD0ZisUEjH+t/s6etWNwNu9a0FvsmDy3OPQmSzYNS4K6hZPAguGeN5J3F6SJKlneVEXUJnkxvaPmHzgHCnqDIOGnaDiSVUSGVUtvtFc+M+vpLRXLAMajeYYU3EYC4C5/YAn9wHB1Xgc8ftocNx0Ig2Bdbns8IXWYgERGOvoMq17kaqerGvAxTUsdmr3o6Jl8ciSzxdcHUi7CCjdqBHSL5B0WTHJtfMU3Qxo8zxn0DXJvE3qxTnz0DosHN2lTIzdwNd3YQ27ZHbEtAdaP8eL10m7aC6ZdIivrdUzQERDwPdu3WL/HgifxQcTCVmFCPMt48VJbtJ998exI1QlQkqBDRarDRLxXRa66hCByCkFwmfxIUZBOpsKx2Y6xldCajAhMrj6/XkOiZS+cPDcjAtUK/DJgLpoVSUI3x1KQeyj8xG9aSwVsnbU6AVb7QE4nWxAtdE7EGhIgWj9S1zz+/xAUuXSOgAioPFoJjouGMrGBcBRnqBqrC1STjIowL8i9IPnQ5Z1AeLopsC29WwWnZzDkAI74RDRkH4ua5/lORv4O/dr0LA2ubSWQQdiMc2X7V59h/8HDF8EnFtGZVLqOSBhP8TVe8DyN1Tcf+tnUe4NjF4NbHoTuL7dcXtMe+Cx2VQznZxLNTBAJVLn94DEA8D5Fbxflwv0/QVYO8V1XKpmb5Jn2z9i/eYb5ahpDk8HBv8B6N8HDHlFYREFwLJxfC9r9OLofEwHpnXKvYHNxWLhASaHhdUG/s/eWYdHcXdt+F5JNu4egiS4u7sXKFCKFgr1lrrLW3nfyld3pUUqWKFIS1vcocVdQgghQiDuupuV74+TZbPJbpAWCnTu68rVsjM7M7uR+e05z3mes/sJ/Xk8I+/ZRoLehLryfuDpqmJ0mzD2phTw5c3hNNv/X1x2rjn/dEu9nriP+ILy6ED+TMghwtedpmE+DG8VRmJOKR+tO2l3upMZxQyrdgkms4V5O6XYk1WsJ6vYPlntuz+S6N8khInf7CS/tOL84waTmU82xBPu68aETlHMmtqJ9MJy4jOLCPNxo16gJ+G+l9lAUVBQuGG46oWcl156iaVLl54v4gA0bNiQ999/n1tvvZXTp0/z7rvvXtEocm9vbw4ePEhxcTEbNmzgySefJDo6usbYFcALL7zAk0/aDHcLCwsvqBT6N2E2WzCZLDyz5DDpheUMaRGGTqth+vz9fDwsgnZuaXioKqQ75ebjPAHBzVc8aAA63AWo5aYf1kIKMdVRqWXR1vRmeW6ne6DPC5CwXs5Tv5eMPm3/UFKXGg+xyah13jWVJ1YMJVCUCd8Ntz3mHQYN+8OuL2WRl7pHHh/xkSw2frxNFiDlBfDTNPGxGfstHPsZTq2TcS2zA2WAizu4+0q0qiOOLpXu45+f2R7r85woiazvFUihZ+Mbski943e59qoKn6hu0g3d/qH9YgxkJMxiljjXkGYw9E0oyQMX3TWXVKX8Ll6bJGWX0CbK7/KeXJAinhVXgCAPNUYzZJRaiPC61EJOkFybgkOU38XrFJNRFCq7v7F/PCsOvh8J92yoPf3wbybE2407utdnWKtwsIDh9t9wLcuW+5dPOHgEo/HwJ6yiBHdDOar5d8m4Fcgodp1O8qG+6QgZ8SnLlXvkmhdg8P9JEmPaIbmXDX0bo08U+lvn4lGUgkrnJeNYnkEyrnWsmnFt2kFY9Sz0e1ESJlUauS5XT1lDJG6Vr+qYK01579sq41yHFkC7yYAFl0stKF8EV/R3UYW8l47WDSuflmJaVU+++DXQYjQcXiTJozEDJRyi1ViYtBiyYmXtEtZa3t9fH5Pnxa2S7+XxykJOeYEUh/q9AG7+Yopv5dwBGelqPR7qdpOmm85bmljV11lqrXjwABj1WA7MJ7jbc1CcgSX9GL0Pf0cvtYqS8Y/iuetN1KfW2D1dlbwd3S/34dl3Bh3r+nFz63Dm7kjilRXH7IouVlrWqZlKZrZYKCgzOH2L/TxcOJya7/B4AB+vj6df0xBCfdwI8tbRMvI6Sz5TUFC4olz1Qk5aWhpGY01zVaPRSHq6GKBFRERQVOQgCrkaQUFBaDQaMjIy7B7PyMggLMy5ikCtVtOwYUMA2rZtS2xsLG+99ZbDQo5Op0OnU2ZOa+Pw2YLzhnCRvu58tCGeL0dG0uv4/3BN3CAdne0fi+R4g5POUbObpaM19G0pfJzbC+telljQsjzxeLEWdNx8ZRb+9GbZx1Ai0eJ9/wOl2dKhO7wYhr4rRZXkP22mxAAx/Wp/QRqNJGA0HyWKGzdf2DPLlpxgZdObYuY34GWZ9c44JkUZz2CwABtfk/1WPy8z9wfmyoJFrYFW48SI2WKSufwmN8m2qhxbDneuFsXCjs/l2Kl77Is4Vdn8poyGPbBduppFaVKc8Y6Akkx7xVJVkrbbjJVdveTrGkT5Xbz2MBjNnM0vY0iLy1Bt6QvFONQr+O+/MGwR5GeLzER4XeLolmeIzeNCoQbK7+J1SnE67PjM8baitMqEpatXyAHxR7QZpUeAX02/nCiPCtT7F9iKOFZS98jXmT3SzLGYZOR42m/w84MSdGBl2wfox/yAq4c/qnk3w8T5UqCYMA+W3u344nJOycjy6K9s491dp0OFgxHxqlSUwexBogbp+QT4RKIqL8TPtRi4+Ejoi+GK/i4aSuDUBsfbkrbL66zqKah1A32xjJdbzKKYaTwYynJkFMvVG7Suogqu+r3UeckoOIgyp8dj0nT78TaJnG8zURpfFWVSBBr4P9g7R8IetG5S1Jn0o4xnVfUujOokyaKVeGQdQqcuxLL8IVSVxSkV4N12IsQ5GM8H1Km78DblM+zbWNrX9ePWDnUcFl2CvXQ0D6+prnbRqBnZNpL1sZkOj39TyzBi05x/3kkvLMdQzcNHQUFBwcpVd17r168f999/PwcO2JKGDhw4wPTp0+nfvz8AR44coUGDBhc8lqurKx06dGDDBtuNxmw2s2HDBrp163bR12Q2m+1mjBUujZ2nbYoZH3cXCssMtDLHShEHpLtybr8siKL71jxA90ck/WDk59LdmT1Iihfl+aI0CWkON70nBRCQYs/GNyT5qixPFgTxa+HboVIs6v6omO95hchzqhZxQLp0ztQmPhHS3Wk9Xsz9vhsm3joqlUiJtVWSeUqyZLGy6jk55p5ZspBY/Ry4uIk0G2RxvGiKFEhGfQ4jPpYkiNTdUrwJbgItx8kok1tlt8U3Cm75RgyN49fKwmXo29KJckZ2vMy0+9aB6D6y+AlvIx+UHXkT2b0nxbVvV1BwwNn8MsyWy0ysyj8j//W8MsqvYHcp5FxeBLkyWqVwA1JRLvdjZziLmf6H0ZRloXIUTw5yv+vxGBz+EZbcIffs5fdDz8fs1xtmI54/34mLsUgaFxWVI8sWs83w2BEFlWrb0mwY9LoECJTmOR9D0/nIesFQDGd2SzGiKB0OzMNdfYGEyGuN2n5WQNZopirvXeuJosppN1U8/k5tlLF5j0DxScxLlJ+x6gW5VuNkXOvO1TD+B/H1W/eybItoK35IIz6WNdSgV0WVfOI3+b6VF4hiavXzoiy20vk+GXmrkv5pDm6GJvf0+SLOeUyGWqPNteW5uGrU7E/Jp1WEL88MaYybi+3jU/NwHxbe15UIP3eHz+9Uz58GlQbLVfFw1TC8VTjNIpyb/Yd463DVXGcm2QoKCleNq67ImT17NrfffjsdOnTAxUVSToxGIwMGDGD27NkAeHl58cEHH1zU8Z588kmmTZtGx44d6dy5Mx9//DElJSXnU6ymTp1KZGQkb731FiDzxB07diQmJga9Xs/KlSuZO3cuX3311RV4tTc25RUmisorCPez/xA3obUfIcfesj2g1ojEdcUjUpDocIfc3LWucvP2CJYIS2tHpkFvkd1aLNLtKS+AY0vF4C5hE6AS9Ut1jOWw7UPo9QRU6OHAfHAUa7rjCzHXW3af/YJCq5POW26SeNNYMZRI9ycrDvq/ZDPU8wwGLNDpXjg4D/Irz+XqJcWg0V/BtzfJYqeiVPx/3P1FBr7rS4lhBxn76nQv1Ospo2I6L+lcBdQXr5yUHZD8hyiVGg5w/g0JalQzqtWKu7/z50ENI0oFhYshKecvRI/np4jKzePKmGm7aVX46iC12PkC3SmewZWKoQJbcVVB4XrHxU1+np19QA9ucnWv52KJ3yj3N0eq0qFvwy/TJTjAytl9kmw0caGkMeorDYCNevGe84mQ9wLkb1DV9KrqBETLePUt30izJKyV/Hv0l/DDaCnYWFFr4Ka3bfd2kHXM6udFMXy9pVa5+dW+XedtK4AENZaghbJ8GVc7u1fMpXd8LoEMzUaIitrqX2Sl0z2ioio4I95NpzfajukRIB5HOm/wCod7N8mIuMFB4yknQVTbnR+Aup2lKVg1YEKtQd1hmkTCV0fjWhlr7/heUaELwGASWwAXjZr7uoRyR2sPik0uFKs88HN3IbCWtKhwP3fm39OF2dsTWbznDOVGEwObhfLU4CZE+nugVqvwcddSWFaz0PfYgEaEXK4HnYKCwg3PVS/khIWFsW7dOk6cOMHJk2IU1qRJE5o0sS0g+vW7wOhLFSZMmEBWVhavvPIK6enptG3bltWrV583QE5JSUGttt08S0pKePDBB0lNTcXd3Z2mTZsyb948JkyY8De9wn8HGYXlfL7xFMsPnGXBvV2Ysz0JEGO3jlG+cKpyUVSnk6hERs+APd9IuoGbr6QQmCsgeQfc/IktCaHpcImRXFIpde77ghRSTvwui7iRnzsfEQKRSre8RY5XeMbxgjX9COyZDbctlojM9EOyWIjuI8Wk359wfOzkP0RW7e4vZsrdHwP3AJFPl+dLkScvGdrcJsfxDoP7t8o1BcaI6eGx5aLEaTYSxnwjBskn18hXZEcYO1sMmd2thRW1jHgdWy6L077PS0fQ0XhVlwfsFUNV8QyWgtHJNTW3Nehji1yuTlmBFLvcfGpGgir860nOLkGrURHo6XrpT85Plp87zWXEll8kwe7qy1PkWFVCBalKIUfhxsErDLo9ApveqLnNO/zvMzv+uzl3EFqNkft21eZLSHPITbQv4lgxVYhKts0EUWxYKcuXe+iZ3WJWHPsrtJ4gSZXVCWokRYYD8+SeHdNP1MXHlkvD5e61EPsbpB2Q965+Lzmn1fzXSu5pCGgg572e0LhK88jReFX93rI2azpc3seIdnDoR0jeLuNurSeBTxwc/1lSNcd+K+bPWSdEKePqKWNT+kJZG3mFi1JapQGLUdTE/V+GlU/ZlMjD3q99/Re/Fno8Dql77b0F3XzFrNozyHERKGkbNL4J4lbW2GSK6sof6aLufKZfJI3N8ah/fgeXjON4BDQgtPcz4NWaC6WRRfi589zQJtzTS6YNfNxc8NTJR7BIP3d+vLcb983dS2qe+De6aFTc0zOaoS3DFENjBQUFp1z1Qo6Vpk2b0rRp07/lWA8//DAPP/yww22bN2+2+/cbb7zBG284WMQoXDT5pQZe+vko645nMLRFGDlFep4b2oR318RJOoyHLyUtJuHZ60mZld5Wqa5qMVrGnn59FE5vgjodxV9m/SuyICrJgdhfJGbSWC5dM796tuKEUS8mes7MgUEWV6jkvGmHpCh0cnXN/ZK2QVsxIETnI4uzHZ/J+JSjRaGVwnOyICjJkkVkSZYoCgrOSORmaCtJsdj2vvy71QRoPlISu6pGpabslCLXsPdtsZtn98qCdfuHcuwej8lCqvV4WZTG/S6pEGO+sU+tcnGX91Wjc+5v4+4n0uRfH5PFjpXofjLuVV2RU5IthaM/PpbX2KAvdH0A/OpLIomCAhKrGurtdj4F5JLIS75i/jhWAt1VnLms0arKQk7+GQht8fdelILCP4VGCx2myZjQnlm2sZPgpuIVczX8cYrSJAWqvED8UDwC5f5UG63HShrjLV+LuqW40hcxskPNoklVzuwS/7mqhLaQ+/XOr8SDb89sGUM2lEiBxlL59yKinSQwzR8r98jchMrkyjyo1wMaDYTl02XUx2KUsaKvutunTFpRqbC4eKBycz5Cc01ybBkMeQv4j30BpUEfuOkdaTp1e0QKN398IrHrqXtF2dKgpy1JylguyZ0NB0q6WI9HpahVVgBad/jzUyjNxRLdD/P926lAi0rriquhCNWA/4qSKWWn/FfnLWsSB1h0vqB1R1WnA9y12hZnr3YRJZV75YhX2iH7J+6ZDWPnYLFYUJ20eRWa6vakfMQXvP3VSe7qFsUDdZLQzJ56XrmjKjgDiVuxDHkLVctbwds+Qbc6rloN4b41x69UKhXNI3xYOr07uSUGyipMBHnpCPJyxcNVWW8pKCg45x/5C5GamsqKFStISUnBYLCflf3www//iUtSuASyiw2sO55BHX93nhnSmJMZxQR4uLJ0ejdUQFxGCU2aDIRFk+xltBlHpWs19VcoTJXxqKX3yMIo+U+5occMlEWSSi0mvXErZbFWvzckbRWDvbGzZdTJEa3Gi+okqpOMJfnVky/r2JMVFw+5sf/8gP3jKrXj9AMrWp0Y7BWlywKxXk9YWyX2MjteOlCjvpAF5smVYoxYtYhjJXWPFGkComVh2uEOmffv9bTIfLd9KElc+hLp5t23RbqJWjcpJukLRO2k0cLRZTJnrqs5h30enwgpApVkV46M+MhYm0e1sauyPDGerppskh0PB+dLwS2slfNzKPyrSMou+QvR40lXLLHKSrCHisOZTn6Xa8PdX8ZBC878/ReloPBP4hUiBYouD8goiouHKBWuRkphZqzEg1e9H7caD4PfqP1DcGgLMJRKk2Pgf6X5YjHLGuHQAufPc/cX/zorjYfKPd46RrXkLuj6oBQHejwu997iDGmIqDVScLrla0n6OrLIdpzsk5IqOe5bURl3eQD2/QDDPxL33PJ8CVw4V+kFGd0PVdxKjG7+aH3DL/FN+wdx9YQfRkk6Vf+X5b3UeUsTacH4ygjycvn+5CXLe3vLDGlOuXja0jPr94Tmo+V4SdtEudNqHJTmihI6ZScAquyTaA4tQHXHStSnt8Dmt2zmxdH9sHS6F1OFHu3aFxxerqXdZNRzR9mU2Gqt/JxYzDDgv9Ica3EL7P7a3gPNWA5rXyFnzGIK2z+PzlhEhdaLzWctZB0oZ83jPQky56CZOdrh+JVqw/+whLdGpdaC5+WPCof6uF3emLKCgsK/lqteyNmwYQMjR44kOjqaEydO0LJlS5KSkrBYLLRv3/5qX47CZXAuv4yH+jWkaZg3zyw5QmpeKU3CvHk+sikL9qTQt3EwxlNr0VafhQZZAMWthJOrZHbdSsImUZUkbIT938uNOLqv3Hh3fgV9npVRrJQdMo7V9UHY+aX9scNaQ4NeYi5oHa9a9YwUVQ4vlgKLySAqlD7PS4etOgmbRGJ74rea21w8pHtYJOlqtLtdFoLVsZglnWvAK2KUWD3WtCqxv0lBKLqvyL9nD5SFgtYN2t8OZr0sdiI7yvbmo2SsKnWvGAIaiiBmkLyegGjn57Hi7n9hv5yijJrxtCDnXfmM+A5UL/4o/CtJzCmhhYOkjgtSlleZWHUZaVeXQLCHinMlFkxmi6gFLxaVWsa+lEKOwo2IzhN0DaRBcLUoSIXvb66ppjiyWJoM/V4U3zxHeIfBhB/E9HbzO3Lfi+4v98gWtzhv7LSfJgUXd38ptrQYI/f2kZ+JCiM/WUamw1qKQqnlrVKE+e1xyEsStevY7+yLOFb0hbI2ie4DgQ2lELbjc1GyeoWJ8qnDnbDlbej9LCyciLrxULieCjkNB4jp8IJxNbd1ukfWCa0nyJqnyU3w4yRRXne4U9ZJoS2h99Myzr79Q2lE1e8hKZ7GclEWNx9lf1x9Eer1r0JoM/sEqtObUH0/As3kxVhiu6A6s8v+aR3uQ6cvkbH9tINyj6lidCwx8FPALwruXIV510zURxaBSkVps3GcbXgbt/+QdD6BFcRXqU0dX/o2DibINdeWnFodox5VUbp4G/6FQs71QKnBSFp+Ob8cPEdKXin9m4bQsZ6/U6NnBQWFK8tVL+S88MILPP3007z66qt4e3uzdOlSQkJCmDx5MkOHDr3al6NwGYT5Sgf+kYW25LHMIj2TOpeyeE8q93UKwO2Pec4PELcSYvqLvLXNJFkA+NWDVc+KMqfqfqfWi+T7l4ehw1QxDLSYZGHXapyMWpUXyjFMBlh2vxRC/vxMFmsrHpbCTvNRMOpL6bR5h0m6xeD/q3lthxbC+O+l4FRVRaPVyfF2fG77d0WpYxk1yCJG5wOqczaptkMsUlja/KZ9YctYLoUbjQ7qdZcUq453ywdMlRoa+UOTYZUmgKHg/jf6eCRucb4tZYcsdJVCzr8eo8nM2bwy+je9jE6+Ndrbq3Yp+l8lxF2N0QwZpRYivC5x/MszxJaspaCg8NfIOul0JIY9M6Uw4Bfl/Pk+EdDtYSkclOfL2O+CCTIi3f8lSbKsSqMhUK+bjI15h8HG12V0ObId7J8nChOfcFGZnN4iaUud7oL5t9qCF8LbiBLYGQkbJG3SUCRFDKtaIy8R1v9P1ijTfpP/L89HxWUYr/+TaD3F5Pnn++1VynU6il/Q4qkyluYZJB5DJoMU2+7fKqNqdTqJYrnq+Fv8Onm/p1VplgU2lKKbX11RRB1cIM+tTn4yqqx4GP4B5CZiiVsFOi8srSagcfOHU2ukONluiowPrnvFpr5SaUjOLSX5XBYxwUGE9X+Jii4PUGGy8M3eAj79NhGzg29Px7q+RKqyURvLan+v1FqIXQERbS7+/b3OKKswse5YBo8vPnj+R/3nA2cJ83Fj0f1dqRdYiyJcQUHhinDVCzmxsbEsXLhQTq7VUlZWhpeXF6+99hqjRo1i+vTpV/uSFC4RrVrNN1trqllcNCoMJjP5pUbq1pbOYN02eoakGPz2BAx5076IY8VkkOJJi9Gw4TV57NZZsihrcxs0HAQhzURdk3XSlk6R/KeY5o37XswQywukAFSaJeNUZXkyluQZZN/1qSiV6NKbP5Nu3Nl9sgiMaAf758o4FMiC7WISKBI2SUqVs1jXJsNFfVO1iFOVvbPFY2fuaOkG6XykCxnWCn55SMbUgv9mg8rrLVlD4R/hbH4ZRrOFsMuRgucmSpHS88qmpQV72CLII7wu8efaM7jmSKaCgsLl4UgBa8VQAhf6oAziP+cdKmPaBytHqvbOFi+8yT/JfdRYDo0Gy7jOt8PEh27Vs7bf5UaDxddm3UuibAVJypy4EHbPthVx4ML3eZVaitGLpjhOPDryk4xsnVwNYa0wa93RXPhVXjsYy2Tce9KPoqopy5e1UGk2LL1XijsunrLusqqiLGbxO9z0hoyqOfIwMhlg42uSYupXT5TXO76QUXz/+pVFPSdeiGd2wc4vIKIdqg7T4PgvqObfitrdHzreJWNey+6RsfdRX8j4PpDXbDJP/HaW/Sn5eOm0zJrWkfZ1w/DQaujS0A3zpqQap6oX6MHT7Uy4zx8k0fO+dexHsqzofICLXBNex2QVlvPkT4dq/KinF5bz+m/H+XhCO7zcFE8fBYWryVX/q+Pp6XneFyc8PJyEBNvNPTs729nTFK4h0grKqTDVXLQcO1fIzW0iSC51oajVNOcHaD5KTHxPbxLflZDmjos4VhK3yuLBysnVckNd+RTkxMvNf/3/IKSpJGBZE5iS/5DZ9oi2UoBZ/bwoe6xz28eWiWxaV200xNVbijg+UdJt0upE4aPWyGKv5a2yENG4ysw3iKeNq6f8F+T6SnPlGup0cjz2FNFOZscdLQysVJSJuscq6dUXyoInfh10fbj2Dubl0qBPLdv6Xng0S+FfQWK2RI+H+15mIccn/IovfIPcbYWcS8YrWFHkKCj8XYQ0c77N3f/SEp18qpkyH1wgni0nfpO1hJuPFBLMRmnWWIs4Wh3415PGiLWIA7LGiF8L6Yftj5t9Ujz7nNF4mBShck873yflT+j5BAx+A4PGSRjBtYrZAGW5sHiajKef3SepniufkaaXb5SslfJTZT1kxd1PHq8tYSppuzSkUv6EFY/Y/BTzkmDtS3BokYyvV8crRIpnwU1hzhAZsS/PFxXUupfl+9plunjx5J6GyA6Yg1uQEjqQ/Sn5ABTrjUybs5v0AlFUNwv34clBjagaDqVWwawxUbgvu0Magbu+EhV31dcJcg8b8n8yZtZ85CW+wdcX+1PyMTmSLQEbT2SSW2pwuE1BQeHKcdVLp127dmX79u00a9aMYcOG8dRTT3HkyBGWLVtG165dr/blKFwGWideE3N3JPP9XZ15YtFBmo3qildEe1Tn9ktnpfloGQPSF0mnxFAEW9+TJxr14Obn/IQu7tXMh1W27ldFqSh10g7Cd8MlNWHQa5VFFY3Efs8eZG94aKXZSBnNGv2lJGkUnBV1i7rSPLjrdPj2FvtCy66vRD1kMsiM/dC3ZSHnX0+6VR4BkHFcZsO3fyhyYY8gmQk/uRriVsnxm4+SAk/ChtpTuNQa+arOsWVw7+YrM5riFQq9n4Ot79g/7uYrSRUXShhR+FeQnFOKVq0i0PMyzI5zT9X8MHYF0GlV+OlUnCm6jJEGr1AoyYSKcnBRDCgVFP4SAdHg38A2VlmVnk9K/PTF4hUi49kJG22PWSyiGulwByRuF9+4TdXGp5uPknu+o5HoglRRh2QcET+d6L6i1tW6wZRl4reXU0VV5BEAHe+0V/Q6Qq2VEavl92MZOeviX+O1gForI1MTF4hHYUm2rOPKC0Qp3fsZUb94hkgT7efp4lFTXBkGoS9xfmytm6ifNr/lePueb6RxdmCu/XOCGslo90YHo/Eg4/GTf4JdM+DQj5Tf/AVHi3y4e4l9w0xvNPNnQg51Az3x83Dlzh4NGNkmkl2JOZQaTNQP9MSfVNvPa2asqL8m/Shj/1knJLyjyU1wYB6W4GaorsI97Z+kqLzC6TazRcatFRQUri5XvZDz4YcfUlxcDMCrr75KcXExixYtolGjRkpi1XWA2WwhwNMVT1cNJQb7NJicEgMZheWk5JYyZXEyf97zDaqSTOmwHPhBFCoN+ohcV+NuM6I7d0CKI9udfP9bjZeUKyuNBsPq5+T/A2KkiGMlZef5BAQAbnoPBr0hizBTZbdApZICi6FEOig/PyjFIo8gUQh5BsG4H0T6W10tY7FIt2jiQlj5NIS3lVGszOO2fer1lCJR72fEQHH1s1K0aTkO3APkw+HRpbD+vzImZtRLMccalVmVpiPg1Maaj1ssspjSXmZiUG24+0rUeExf+PNzud6Y/jLKVlvRSeFfRWJ2CWG+lxE9btJL0TSy45W5sGoEe6guT5HjWen9U5AKQQ3/3otSUPi34RMBty+Hdf+tNIRVydhNk2HQdhJoLmHoyCNAxmbW/ReOLZVGj9ZNRnKa3CQK3dYT4I7K1EjvcGnWhDSXwoQjjv8iDZcmQ0VJsmC8rWHkEyneeQcWQsZhKfJEdZGRrWY3y2hWogMvHY2L+Mlkn4SMo2i5jAS9fxBVcaYUtQ4ugKM/VXk/ImD8D+I1lBkLxEKjQZIKNvgNCUw4vQ063+v84K0nyBrMUZMN5HtaliffV2O5FJBGfSkx512ny/fTGbmnpdhXXsD+fG9uW+R4RPZUZjHF5RV4ubngXfllMJl4cN5+TmWVsH1KNe/BxK2iso4ZIIqgVhOw/PERls73ow5rKT+XNzAd6zt/fTHBXni7uVzFq1FQUICrXMgxmUykpqbSunVrQMasZsyYcTUvQeEvklZQzturTvDSiOa8uPyInTmcm4uaOv7uNAr14ujZQsotGtwPLUB1oIrxcfZJ6ZjcudLmTxPeRm68PZ+sWcwJjJEumnUcqtFgmc8uzZV/l+dL57w4w/EFF6eDoUwWYRVlEmHqHSoy6t8eg5AWMHaObCs8KzdnFw/pPp343fExzSYZd5qyVCTHVYs4AMnbxeAvsKF0bkBm9w8ukIXitvdtpo9Hl0FgIxjxMfz+pBS9rNTvJaNdP93p+Drc/kaD4+p4BIjJckRb8RJy9b60hbbCDU9idsnlRaXmJoqPwlXqXga5q0gpvJzRKmshJ0Up5Cgo/B24uEOrW2Xk2WKGXk9B3W62cehLwScCbv4Y+v1HijUqNWz7GLZ9JMrc9MPwx6fibXLzp7BwggQjeAQ6HoXSF0pzKeeUNFqqUngW5o8TNYbHdFj5rE1RvO87ua9nnZC4cisqFYz83Kbq8a+P2s370l/nP4mrF5xcI8liVSk8B/PGSgDE/u/kscOL5X1I3Q8/3yePeYdKatXW9+2fHxAt4Q1l+bWf37eOGBt7BMl7mHZQin/dH77AdXuCUY+p4WD2pDv/2x8d7MWxtEK6NJCkqYzCcsoMZh4d2AhvnQuuHvm2QpIVs0nWjxpXLF0fQnXrLFTWEfsbnFAfN25qEcaqY+l2j6tU8PqoFgR7X4HGooKCQq1c1UKORqNh8ODBxMbG4ufndzVPrfA3EZdeyLb4bO7tHc3C+7py9GwBhgozHjotbaL88HbT8kCfGBbvSUFXUWhfxLFiKIa1L0O/lyTm08VDuhwaF5i0EE6ulQJN/Z4ycrX8PklOGPExYJEumJX4DRJFvv6/Nc+j1siCbt3/ILy1fKm1kB0vyiD3QIjsICNSSdukMFJeKGbCzUfVnjZlLSZVL+Kcv641Ej9qTbkC6Txt/0gety5syvJEofDrY6Lg8QoRU2a1K2hdYPO7jmXgvlFXPPEHkO/NpXgXKPxrOJ1VTJsov0t/Yna8/B5ejZ9fRJGzN/0yOuEegfLh0JFSTkFB4dIoTIOld8u93srpTaLMmzBPPLMuFVdPSSkqyYZ5YyQJEyChijfLkmnwwHYpwsSvhfZTbaEF1fGJgOUzHW8ry5P7fUhLOL3R/vHfn4IRH8no1bkDogCK6SfXZTSAUY9l8JtofcMu/TX+k7j5wG4nzdbyfFHrWg2ADcWABX5/zLbPvm9FlVN1XVevhzSKEjaAbyQEN3EcBuERCMVZENxM/g6veASGviVrwvSjonSq6nNkxcVdPJeMeiy9nqJ5ti9Q04cwyt+d6CB3jiRl0DLcm7MFeu76bg+peTbT7RcHN+COXs/isum1mufp+SQq7zBw/fesjwI8XXltdEs6Rwcwc+tpsosNtKvrxws3NaVx2HVWpFRQuEG46mbHLVu25PTpWozhFK4pyiqMpBWUkZZfRnpBGYnZJdzfJ5o/TmVTZjBxNq+MvSl5ZBfryS81kF+qp6t/MbPbnUYT50TRAhJxHdkBojrLTHpUZ9j1tahPClNFwrvjC1n4BTaW7v3ubyThypoq4eopI0AB0dB0uP3xtToYMxvcAmD0V7D7axmJil8HiyZLyoRfXfjzUzj+s5gJ556WAs32j2QBFtzU+fXX7yXKAmdYLPbpF1ZSdsg41vnj9KxMg8iTD7jhrSXJKiBa1Dq9n5aFUlU8AuC2RZe38FVQ+BswGM2czS+7vMSqnFPyQUdzdfoIwe4qMkosGJ2YNDpF4yKdYMXwWEHhr5Oyw76IY+Xs3tpNcS+G8kJbEccRcaugrAA63ycKoHZTa+7T/dHKUZ9C58fJT4GcuJrR2HmJ8ONkOPSjGO3mJcPCSRKRHhANAdGUBbe6rJf2j2I2ygi3M/LP2EZQY/qL4rmiWvrY7pm2dV3Xh8SceMld4nWTvFOadNU9ErVu4rkTGCPFs7mjYMzXskabvARLQSr0fUGKPVVRa2DY+5iSd2O8ewPawBja1fXjtVEtCPJyJSrAnWeGNOG36Z3YcEcd2sZ/zj1nX8b9z/dwL0qivr+9kfH/rU3kYPAozGNmib8TyLpx9AwpUP2LijhWgr113NG9Pssf6sGWZ/ry9e0daFvXHw9XJa1KQeGf4Kr/5r3xxhs8/fTTvP7663To0AFPT3tJoo+Pj5NnKlxtknNK+HRDPL8eklnkoS1Debh/I7RqFftT8rj7+73nHew3xGYSE+zJqon+uM4bLt41DQc6P7g1HqDfyzL7bDFJgSPtoBRbrKi10Pd5WVyN+hLO7JTo0aAm0skxFMOvj8pNvcfjtsQKr1Ap/JzeLGkZXR+CFQ/LQg7khu8RIEocR6x7RZII5o+rqcxpPFRGoDxqSW9Sa+WrNjwCoe1k8etRqUWNo6mcMQ5qKEbK5YUw7TdZKGYckzGssJZXbSxFQcERKbmlmC2XmViVFSed2KtEiIcakwXSii1E+Vyin49XiKLIUVD4q5QXil+cM/bOhqbDan4wv1jUallTOIoAB/GmKzgDOi9Ycid0uV+aIRnHZC0Q0lxGqfMSZczLOvpcncCGsPc7WRt8d1O1EAZE1Zt2EKxNLJVaikMqFaUmLdfbx36LSo3KK8R+ZKwqQQ3h0AJRwDQdbksHq46xXNZ17adJMSamP/jWhQ53iprn7rVS6DuzR9JHGw0GC6K0PrlKjnFwgaz91vwHVevxWNQ6LHetxZz0J5qkLZiDGmNufgvluOIe1R2tiyuYKvD3cMNFreLTSe0orzCz4sAZxgWexvWH284329Sn1hOl/YQPRi9iWqkXJ9KLz1/6lAXxbHlmBGF39ballXpfHTXptYpKpbq8sWoFBYW/nateyBk2bBgAI0eORFUl689isaBSqTCZri8zuBuVM7mljPnyT3JKbHGCKw6lsf1UDj/d341P1sfj6+7CzW0iCPXWkZRTStcwC66/PiQjRBlHof+Lzk8QM0CKLJ7BENoC4tZKwSZho6RK6AtFgjvgv2JevO5leV5oS1GohLaQok3OKSnghLaUGfXCs7Bnpr1JcWasKHQq5bYAeATLAsuZv07mcfGFmTAfdnwGZ3bLtXa8W4wNVz8vC5d63R1Hp7cab1+QslKno6RaNR0Bg14VWXhVSnPlGnVeUuCyzl4HNJDFj4LCNYA1ejzM1/3Snmgolt/Nul2uwFU5JthD7jNnisxE+VyiCNUz2PmHEwUFhYvDYhHfOWeYKuSD++XiHgANB8noVHXqdoUGvUR1+/ODUlT44xNRhvR7CTJiYf2r0rDJjhdz342v1zyObx1ZRwTUl3XDtN9g89sSn+3uL2EAke0l/MBK64mQGYvFOwSdW8RfeIH/ECaDvB8bHIwW+UTK2HW9njKOvvp56Pei+OoYimvu7x0uKaM9HpNxunljsEz9BZXOA9a8IL47fvVsQRA3vWuveDm7FyLaybpx7xxUe+egbzWZH4MeYcjAcWCuILgsEa91T6BK2i4Fl1bj0fR9jr5Ngtl4IosXfz7KjxOiCFkzvaZi2qgndO2DvNB7EdOW2K5fbzRTrDdCyL+7eKOgoHBtctULOZs2bbrap1S4RExmC8sOnLUr4ljJLTGw/EAq745tjb+HKwt3J7NkXyrNI3y4Kdod1h+RHc1GUaR0vk+UMVVx94cBr8C5w2JurC8S9U5UpVz5pndEWpubJHP11iIOSIEo46hEeU+YKxLn3NPw6yMi6Y0ZIAZ8q5+3n7vOOCYyXRd3KQ4FN4Wic6Lk8QqWMa6qBRmVSlRCKqDXM+DmLR29Xd+IiWJYSxnBGjtbFi6n1sliVa2RIk6X+2HWAPvX7eIOg/9PDP7aTJTIciuluVIs2vqOJPpEtIW+/xH58b/ESE/h+uF0VjHuLhr8PS4xpSIrDrBIN/YqEeSuQoUUci4Zr1BI3Px3X5KCwr8Ld19Rn57Z7Xh729tkXXC5uPmIgvXcAZuaxppylHMKFk6UZMyq0eemCkjaKupeq+o2+Q9JverxuIxjW8eEItvDiE9g0xsyOqVxgYML5T496FUZpdo7W0aprAQ3lTXHoimoWk/As08DIOTyX+M/gdkkBZHez0pDy/p+RLSHkZ9KISdhI/x4m2zbMwsGvQYrn7JXR6m1MiplMsC5g1JIG/6BvMdeoXBqg+yXGWt7zsqn4d7NcHqLjLx7htQY83I7/hMj7nyCE1nFdPXOQjtnoC2d1GSAg/MgaRuBU3/j043xAETpSpxHxheeI9qj1O4hb50Wd1cl6EFBQeHa5KoXcvr06XO1T6lwiWQWlrPueLrT7RtPZNE8wpcXlx/llZub075uABpTGZiqdGHcfKVg4REIY2bC0SXSEavbTaI7N78tapbsk7L/gbniVTN+ruyrUovqZucXzi9Uo4Mt70qnxsqBuXDiN0miWjjRpsDxCoE6XWRxl7gVNrxqe46LhyxK1C7i3QMi7Y39Vbp2TYbJ/Lw1PUrjApMWSUrDT3dCx7ug452yMNS4gHeEHGfkp7LYK8mEuj2g+c3i09NwoHjjGPWiBjKUymu2pmCAJEXEr4XblkCjWkbUFBT+AU5nlRDu62anqrwosuKkMOkZdGUuzAEuGhUBbirOXFZyVahE6Rr10o1XUFC4ZIwmM9qGg6QxkR1vv9G/gYQLqP+iZWNgDNy7SZoqJ1fLOPW6V2TUCaS5VJ3ErZJqtf97UegCbPsApiwTRbCxTO7p6Udh+QMw5hvxxjv+i0Sob/9IGlXDP5BClcUs52kyTLxUVjwixzy8CFWPx2qe/1rHYhG189l94mWDRd6PzFgpipmBqoEWiVvFE+ju9aKMzo6X57eZJOu0PbNkLF3nDZ4hqCLayRrO2bmPLpG13A8jod3tsOwe+31MBjLzi0nNr0C1701bEacq+cmoz+zCSxdCBnpUFgc/B1VQVRulv79PNCHeyhiRgoLCtck/4k61bds2vv76a06fPs1PP/1EZGQkc+fOpUGDBvTs2fOfuCQFoEQvxsYlehPeOueddm83LXmleh7q15D0glJG1TWg3fYeLuo+tgSDoEZwbj/8+Zl8aGs6AoIay4Jox+dyk249wf7A+iJJkGo4QObVGw6Cm96D9f+zT4kAkfWW5dkXcayU5YnpYItb5L9aN+kghbWG2BWSlFGVilKRXE+YKwWYiPbQ6R746Q7ZnnEMsqp0ikwVsPVdUeNsektez47PpfDU9wWRCSf/KQkYLW6B9ndI/PjJddD5fpF5mwwSW7rp/yQafdsHNV+HxSKpXvesE1mygsI1QkJWMWGX5Y8TK4lrl1oA+osEe6guX5GDRf6mBcb87deloHCjUmowcja/jJ/2ppKUXcKAZiHccttyXGKXozowV4oebSdD6/E1Df0vF78oaaw0Gw2nN9iKOCCJSd7hUJRme6zREIhbLbHZm9+W/bs8IIreM7tqHn/RFCksLJkmxYo/PhYlys8PwpSlUrRQa+DITxJiUJWc07JGuJ7wCJS1l6NRM7VGCmfVvYm8I8BsgZ5PiaJHXyhqHpNBxqq0Ohk7LzgD3mHO/YhAGl0VpTDsfciOq2m8HBhDUhEMinZHs32r08NoT/xCq4gnSMgqJlflR4SLe01TZgCdN1lmbyAPnVbN3T0bMLx1OO+ujiO7WM+wlmG0quN76SPF1xjlFSYyi/QcTs2nuNxIu7p+hPi44e/heuEnKygoXFNc9ULO0qVLuf3225k8eTL79+9HrxfFREFBAW+++SYrV6682pekgCy61hxL56mfDvHN7R0Y17EOO07nONx3XMc6JGaV8NWWBHY/2BD374dI8STnBPT7D/zykHS3rEkEJdlStKiKVuc43vvkKkkD6HIfJGyCY8ug2Qj599qXRCYNENVFum7OOLkGBr8Ox1dIgcYrDApS4MB8x/ubDGJqes9G6TgVnpW0q5IsKQa5etnvn7JTZro73QPBr8gHvqRtsOZFm6dG4TlRDeWnQPxq+SAY0hySd0jRau2LUuTKP+M86rzgjLy3SiFH4RridHYJ/Zpc4piA2STeUw2uvioz2ENFymUpcipfY16SUshRULhI9BUmNp3I4uGF+89/xl97PIM33V347eG7iGozUR70CJSCwN+JoVTMhq0muVZ2z5SR7l8est1v63UX1a3FLP53vZ+WJtHmtxwfOy9R7t2RnUR5M+oLWPGo/L9RD7ucRHUDJje/f6Zz+lcwFMHOrxxvM5skbSyygy0GvPFNEN0HsipH2be+L542Vk6ugTqdRcGUHS9+gXU6idGxI+p0gthfsfR5FlXSthr+O1k9XmXtSRUNgowEuvk5TdgyewYToRG/nS/2FPNWz//it+n5mvsNeZvQyHr8/khdvNy0xKUX0e/9Lee3Lz9wlphgT+be3YUIv+uzmFNqMLLpRCaPLzpIhclWgLu5TTivjGhBsLeiPFVQuJ646vHjb7zxBjNmzGDmzJm4uNhUHz169GD//v1X+3IUKskq0vPdH4l8Pqk9gZ6uqICbWobV2G9Ii1A0KhVqtYpBTYNwOzxXCg0g3jHx68Ug2DtcjOmcLdJa3FJzoQXicVOaDQsmyALr2HL4/UlZfA19W8xHQcagtLXcSLWu0h27a7XMZJ/ZAe5+tUeLFp4DixF+fkDGshZPFe+crg9KAWbstzDue5kBD28rxZ51r0iRKuVPiOoqC0P/BlC3O0xaKHP/82+V17H1ffimF/hGwB8fyYKx+6OVXf9aUCnz2QrXDgVlFeSWGIjwu0RFTu4pqCi3xbheRUI8VJwpugw3Vc9g+f1TDI8VFC6azCI9Ty4+WCNEqqCsgkcXHSJP5SdF0r+7iAOSsJS4VbxdqpJ2SGLIJy2UoAL/BlJI0rjKvX/j66K4KThT+/HNFTDwf/D9SBm/nrgAhlb6+kU5MXF386PU429SHV1NVJraFTP5Z6DX0zDkLWmCDXwVPILgyBIoTLcv4lhJ3S1fwU1kTTb0HfEaqo5vlIy3lRWgOrgQy+5ZEkEOENSIrFHz+ep0MGUVRtYkmclrfU/NY1RibncH4zpGEeytY1VsLj8UdyLjlsWY63QFzyAs9XrAHStRN7uZiAAfWkT6YjCauW/uvhrHSsgqYcaWBAzGy2gMXAOk5Zfz8MIDdkUcgF8PpbHqaBoWZ8lvCgoK1yRXvUEQFxdH7969azzu6+tLfn7+1b4chUri0ot4sF8jnl1yiN6Ng/HWaWkZ6cvIthFsj8/GYoGejYJIzilh+6lsKkwW+kRp8Tq+xv5Ax5bJoqj/i2JON/Jz+w4YQHAzaHkr/Di55oX0fBzm3lJToVKaI+NMHe+S7k3bSeKjc2iB4xfUdgokbRezu5Dm4s1TkiPxoVZVT3WiusDKZ2xRm2qtSIZdPOHgfNscfVgr6ezpfGTsyr8+LL5dxseGfwSlWbLA2f+9SKyrYjaJXDioMbSbImbPvZ+RRaCxvOY1hbaQmHQFhWuEhCzpiIZfqrw8/aio3XyuXvS4lWAPNdllRsoqLLi7XMJYl1ojHzjzkq7YtSko3GiczChC7+SD7oGUfPJKDfh7XqExjopSMTEe9Jo0UKpiHa0e/jE0vRnKCmQtUtWfTq2R4o4jvxUAVHL8MTNg6T1SHPKrKw2Z/i/D8vugqIrHoNYNy4R55KgCMBTpCbqeFA9aVwhvI0UwR9TvCds/hrJK/8O8REkP7fWU+Ac5wjMYAmLEA/DoUin6tJsMfV4QD5yKUhl563SP+BB2uReMelRb3saUeRLzY8fZeDKHdzfmkZCVQ5NQb9pF+RMXMJD29dbjmrzF7nTFvV4ij2DiMopYfF9XdiXlsi8pj+X5DRk2dA65BQWEBPoTHBSKi9bW215zzLlP5OK9Z5jeJ4bw61CVs+LQuRoFViszNicwtEUYIUq0uILCdcNVL+SEhYVx6tQp6tevb/f49u3biY6OvtqXo1BJmK8bk77ZSYnBxNpjGSy8rysTv9mBTquhfV0/AJ5dcpjyChOzp3Xkvrn7aBkQIjHZVen+iNyk1/9P5LOpe+G2RfLf0hxo0BeCG8P+uTUXSuFtZQzLUUEDJPFi6LvyvKV3w+A3xUTv0EL7/YIaSeTowolSEMqKE3PC8XOlk7ZoSs1jB0TLV6d7pNgS2Q7KC8Ubw1gmqRUJG+RY6UdgwXgYMwvO7JEPpqYKiSUvz6scK/MRD6DWE+DIYlmQWCkvhA53wKLJcq49s2UMbNWz9rPmrl4y3mVVISkoXAOcyixGBZeuyEk7LEVPzdUfMAitjCBPLTbTyP8SVQBeIZJKo6CgcFGUGUy1bjeZr0DXvyhd1DS5ieKt5xkkDZ2D8+z384kEnYckLVksMGGeqHczjsr24yug/VQx5q1Os1GiInHzFbXKtN+kUZVzSkaoVzwMU5ZLsynzuKhKmgwFfREeOi1G83Wm4lC5iMpm7uia23wipUlWni9jUqc3y2jViodh6grnhbCRn4rKOve07bG0g7L+u2OlFM0Tt4giGmS0PusEBESj2TcHdbtJlLgEkJAlyqm4jCJiQjx5dHkiL/Z9jU6dCvFOWYfZxYuCuoPRu4dw57wTpOaVodOqmTGlPb7uLqw+ms7bq/IB0GmTWf9kH6ICbHHnBWXV4smrUF5h5kr8CF8NknNLnW7LKtZfmd9NBQWFK8ZVX1Hfe++9PPbYY8yZMweVSsW5c+fYsWMHTz/9NC+//PKFD6BwRUjJKaWkcvHlqdNQYTQz545O/Gf5EbbGS1RjVIA7zw1pxZw/ktAbzSw6Wswt3e8l+Ox0OUj9ntKV+u1xmW0+sRKaDoM9cyTq29ULUnZBxzukA9N4MOydI+NOzUaKSV7Vm7sjClJsnZ6Dc2HYB9ByrKRVGYqhxRhJxfn5AXtVj8Us1zV1hRRHNr4uyiGVWro//V+EeWNELTP2W1j7ssizrXgGw6jPYf2rskCzWMSgeOD/ZNFxyzfSSZo1UMwUtW7QapxEmfpHS5ypNZrUUCKFLHPlYvfUelkcTvpRTJ4LUkVS3HKsmDdeTUwV8v3Q6GoW6RQUEEVOsLcOnfYSCiImA2QcgZh+V+7CaiGkspCTUng5hZxQ+9hiBQWFWmkW4eN0Wx1/d3zcHYQpGA3iyaJ1k3v4pZCbCAvG2SdiBTeR+3KjgXB0WaXSY5CM/vz8oK1p8stDcPOn4nFz9Cfx2OkyHdwDxPNGXwgu7tBuqqxZ5gyR+yTI34ZbvoZVz8i581OksPHHJ+BfT+LMU3ej6vE4bloVqn+giP2XMBRJU2rUF7DlHXl9KhVE95e1T/pRGPBf+HGSvG6dh4xilWRK8md175v6PaX55Widl3ZQ3q+9c2xppiq1rIfSDomKOXU3KouF/k1DeOfWVry7Oo6cEgPvro7j00nteP2347ycW0bziGG4aTXcHBBEcnIxqXlibKw3mnlk4UHeG9eaWdttf9P1RjN7knLtCjkDm4Uyc5vjv/vdogPwcrs+R977NQnm5wNnHW5rE+WnRK0rKFxnXPW7yvPPP4/ZbGbAgAGUlpbSu3dvdDodTz/9NI888sjVvhyFSs4VlBHqrePlEc1pFOqFxQJqlYrZUzuiN5qxAMfPFfLJhnjiM2W0IiGrmNzgzvg3uglt/CrpYq2uNJAry5UFwOKp0PY2GYUym2RhsO87WPdfifKs200UPCHNZBSp6XDnF+kbJZHmVhoPleJMn2eg34uSHOUdCvNudfz8kizxunDzk5luqIwLD4dvh8mCrfUEiFtpX8SxPnf5AzD8Q1hSGUOeeVwWGgHRMgI1o7ttgWcsl+JSboJEkQY1lGJQSZYs/s7stD/+kSXSCWw0CFpNgDYTrm6yj9kk782e2fLavUKk+KSMdilU41RmMeGXqsbJipUPSoGNr8xFXQA/NxUuai7T8DhM1IAKCgoXRbCXjmnd6/H9n6JkaxTixV09GxDo6UqApytmswWT2YJGrQKTUVQYu7+RD/7e4TJiHdwMPPwvfLKSLPhpWs1Y86w4+P0JGZ1qORZ8I6XBU32/8gLY/4MYHXuHyz184Xgx8b3jNykSeQRKIXrBeFsDBqTxs+xemPwTfHsToBKljn892xoiKw7LgP9xLK2E7o2uM3WtxgVm9oWgJtDzSfH8U2sk7OHkKhlfH/Sa7BfZQf7G63xlnRbYsOYoe3TfmuPmVYlbKWbJ1kJOTD/52+sdDjmnMDe+iVyTB7lleno3DqJzg0BK9UbcXDUEe7kyY0oHCssrMJrk5+vrrQmsOZZhd4pivZGiciP+Hi7kldpUN2azmbSCMgxGMzqtmkahXnSq58+e5Dy757toVLw0ojm+7tdnwlPH+v6E+ujIKNTbPa5SwX9uaoafklyloHBdcdULOSqVihdffJFnnnmGU6dOUVxcTPPmzfHyUrr//yQd6vnTr0kI3/2RxDNLDlNWYSIm2JOH+zVkV2IuTcO9aV/Xn27Rgbhq1QR4unJ3zwYklpvIbPEKrTs+jI/WiKo0Vw6YkyAjTmaTpEXsnml/wm4PA2r5kJSfJH4zgQ3h8E/QZqIkRYHIdyM7SEe/xWhbgoJfXSmgGIrlOOYKOLwIYvrX/kJdvaTbN7NSGdDsZnAPtJkgNx0uCzNHlOVBRYks6kpzJHlLXwS/PyXFmoGvyuvsfK9cn8VS6YfTEP78UhaUu2aILFvnU9N42WQQRU5Eu6sez0xmLMwZLGohK6fWi5Fh90fB3ffqXo/CNUt8RjEta+m4OyR1H+i8pdD6D6BWqQjxUJFSeBmyca9Q6TaXF8jvroKCQq34uLvwaP9GdKjrz5b4LAY2DeWN32M5my/KiABPV968pSW9GwXjkXME5gy1jVSnH4b4NTDgf3IvvZAytCjTuYfL2f0SXe3mJ4Udz+CahRyNi5zHzUdSlYorP/jHrZJ49CV3yFj22pftizhWSrKk2DPue1H9pB8RHz1rIUelArWGnUl5118hJ2m7FGfSD0vTrCoeATDodRltH/quhFS4esC4OZIOmrgFRs+As3tlbWY2SXFOVUvGikptU0ppddD1IVmPjf0Wds8kZ8An9PpwB+UVZgI8Xbm/dzT5pRUsO5DKf29uQa9GQUQFeHAyo4jBHzmPI88rMeCp05JXWoFKBa+NbEluaQUjPt1OTomBYC8djw5oyMcT27L8wFm+/zOZgrIKejYM5JmhTYkJvkTF2DVEpJ8Hi+7rxv9WHGNLfBYWC9QP9OC1US1pGub9T1+egoLCJXLVCznz5s1jzJgxeHh40Lx586t9egUnBHi68tTiQ+xPyT//WEJWCU8sPsSH49ugAkZ/8Qf9mobQt0kIId46dFo1+goTFbpAXtxbzn+6uhBR9aDbP4ZbZsAvD9tFRhLdHxoOgt1fQ/NRENpKfDOW3yfbh70P4e1EFWIoltlrzyDwDBVD4brdRLa79V0Y8YkURHZ8IUUZlRr86oFvHXk8/YjtvN5hUoAxVelEBDQUSW9VjPadCjuK0mUBU5pjn7wVt1LGxYa9Iwu+rLjK40fDTe9AaDMpSo34SKJK206CXV87Pkfjoc7PfyUozZWZ9apFHCvb3pfCmlLIUQD0RhOpeaUMbnGJBZmUHRDctPZF/BUm2ENFcmHt3h0O8a5M78tNdJyuoqCgUINALx0j20bSPMKHYZ9sx2CyqeFySww8MG8/+55sh8eKRxz74m18TdYHtRVy8pJl3Lo2SrLlvm2xQOf7JVHy0AJRjdTvJUWcXV/LyPeoz2W9Yi3mJP8paxWzSe7b1QlrJU2Z0hwZR07+UxoxbW4TX7ySLGg6kgq1GyNaX4cfknMSnG8rzZWmVu5pKbikH4GNb4LFJGux6L5SzNHqpKCTdULuA01HSOPIEc1Giiq48VDoOl3WYkPexFKajfHuDTz7ey7lFfJzlFti4K1VJ/jfSInMfnD+fubd05meDYPxdtMS7KUjq9jxWi462IuMQvmZG98xirj0Qubtsv0cZRXrefmXY2QW6ZneN4ZxHaIwWyx4uWnxdnMwFnidUT/Ik08ntSOv1IDRZMHbXUuIt2JwrKBwPXLVV9VPPPEEISEh3HbbbaxcuRKT6TIW1gp/CyV6I2fzSjmbW0Jmod6uiFOVTzbEU2GyYLbAhthMdifm4KnTMGX2bu6ftx8zsD0hm98TKjDV7WF7YsoO2PstTF4ihoIjP4O710GrW0W6HLdSUpv0haJEsVjka+XTMt/+xyew4hFJNtg7B+bdIp3xjndLl7zNbbJQ2PI2HF0i3TNXr8rCSQtZTEz+SRYFai2M+Fjm78GmeHH3l4WXlYoyecwZATFQmCYmfy1vhWM/27adWAkHf7QVcUAWOQsniQlg05HQbpoUqNpNFaVRVVQqieJMP2r/eEWpSM/Tj8jx9EXOr+9yKM+HM7ucb0/+8+89n8J1y+msEswWqOPnceGdrRScET+qkGZX7sIughAP9eUpcrzD5b+KT46CwiVhMplZsi/VrohTlfLCbPtmS1UsZji33/nBC9PEhFeldq5gVamlqWMyimondbesS3o+KWEMfZ+HdS9LkAHA6hdkVGjKcjFMbnGrqE7UGns1XuMhsrZodrOsNfyiRNmjUsPP0yXF8pYZ8rej6wOozBVo/sEi9mUT1dn5tsAYUVn2fk4CHBZNFmPpQwtl/Pz4z9DhTtj7naRa/fa4jL+Ht5Gifo1zdZVt3R+W79miKVJAO7kG1ZK7cJk7kld6euPmYv8+ztx6mkmd6gLw1soT5JToCfV244VhDs4B9G0cTHJOCRUmCyoV3NIukoV7HEfOf73lNDnFBkJ93Qj3c78hijhWfNxdqBfoSUyIl1LEUVC4jrnqipy0tDRWr17NwoULGT9+PB4eHowbN47JkyfTvXv3q305/1qyivT8cSobD1cNZQYjZyrN4ByRnFOKp872o3JnjwY8u+TweXf7t1ae4OMJbZmzLZFBvd+h/m8ToPBspc/KYxLtmX5YRppMBuj6oJgJ/vExtJ8mxZCqJoAN+kLCxppKGRDlTVRXmdF29xUDvDO7JC7UYoa1L4l3jRW1BkZ9KXGY296Xee1b50jn5/gv8u+2E2U+31AMBxdIJ2jTmzXPHdJCTImHvy/duaX32iczWMyyAGk2UrxuUEnUaeyv4v8z/CMoyZDCTnk+DH5DznVmt8i6o7qIBDx2he2YRemw5V048IPM7qvU0Hw0DPk/8LHTP105qkfBK/xrOZkhRcQ6/pcQu5q0XbqygQ2v0FVdHKGeKracMWO2WFBfyuiizlsKxLlKIUdB4VIoN5o5dKbA6XZnEeXncaSCsZITL42NlJ0ScnB0ac19Wo2FrJOSlOnmI94rDfrAuYNy7/cIFLXIkZ9EFWQsh+X3S0PFrx4sv0fWDv7R0P0xUQk1GgRNhksqpnXUavc3UpwY+Rns+1au69hyuG0x/PEp6q7TKTJf4jjqtUBwE1lnFJ6rua37YzJWXnhWXmdwE/smVsJGGc+eOM/++b89LubSOafgxG/SZGt3uxTjvr9ZGldW4tfKSDtAURoRBz5mQpv7+H5v5vldzuaXnY+yP3auEH2FGbWnigHNQvlqcnveXBXLmdwyvHRapnWrx9Tu9TFbLPRpHIyrVk1WkfOkJoPJTG6JgVAfNwxGM+6uGvF1UlBQULhGuOotAq1Wy4gRI5g/fz6ZmZl89NFHJCUl0a9fP2JiYq725fwrMZnM7E/OJTrYk4zCctILywnxcV6Rd9Go0FT+pOi0aowmM8V62wIrIauYZ346TIf6/qQQSsntqygduwDTrd+Kuub0RpmfzkuSG/ral6SDFdZaxo0SNkCdjrYTNhshkd3OOLlauuMB0dJpc/OThdyhhfZFHJCF1i8PSeHk5BqRCv/xEXS5XxYPyX+I9PrWWRJBmrhFVDt9nhPJNMgCo/FQGP89ZJ+S6//lYTlmVVqOlQ6dRwCse0X207hKGlVRmhg9l+XJwseoh9+egJXPQMYxOLURFkyQ/285VtK9Di+WhePe2TYTZYsZji2T81v9iP4qbn5Qp5bOW/2ef895FK57TmYUEeDpalfYrR2LLOhDW0jH+h8k1EOF3gSZpZeoylGpZLxKUeQoKFwSVtNYZ6SUusp4kiNUaikilORIEXXnV/D709L4Kc6As/tkP88gGf9tO1nutyCF4/bToPczogyZPViMjovSZHt4a2kuLb1bzHwzY0VVMmaW+Oj8+YkcNzdRlLZxK6VQ0fcF6PyAjCJX98vJOiEeeS3GyL+P/CSNJL86qM0mAjyus8QqAFSiTqpXZQ3gESiNpOyT8pr1RaLA6f5ozacf/lFSpwIayOhZn+eksVaQKn45g14XtbJ3uIRJVC3igKylDDYFsi5uOaOa2K9VPV01VFQqvkJ9dOcLLb7uLtzUKpylD3RnyzN9WftEbx4f1JhQHzfCfd1pFOpNvUBPPC5wL1Or4Nklh7j7+z18vjGe5JwSzEpEt4KCwjXCP3pn8fDwYMiQIeTl5ZGcnExsrJO5WYW/lcwiPXUDPbnj291kFxuY0rUu/ZqG4qpRO5RAD28VTmGZFBI0ahUVppo3saxiPR+tFxPBnx7oxv82+LBguBbfgf+z33HPbCmW/PEp9HhUVDAmA6hdxBw5O14WYY78WqwYSmSfjFho2B/GfSdJEfPGON7fbJSFmvX4sb9Ci7HSaZ84H1DB5rcksSrgeSmWaN1g3LcSQVqSLR/mEjZBSFPpIFWn4UBJxZg/VhYpVg4tlK7U+B/kGjU6uGuNjHB5Bosa5/cnbPu7+co1LrsHJi20mT5XJ2GDzN//HYlSHgEw4kOJVa3+vvd8CjxD/vo5FG4ITmYUU8fvEtQ4WSeleNvoKvs+OSDUU6rRyQVmwjwvsYfhFaYochQULhGtRs2ULvWYtzMZR599/YLCYdgH8MPImj453R8VtY1RL/HW1mbGnpkQ2VGUvWot+NcXlWvr8TB2jhRYNC5SBPqmn82fz1AM2z+CiM2SmBndT8a6LBZR/8b0kzCDMTMr05m0sg7QF0lRaN6tMHGBmPc6Uwod/1mef2ihXHdFGTQbjao0E3ff63C0Sl8EOz6DFqPgprelEWbSw77vpQlmpShd1i4qtb2C11Aso7UBMVhuX4bqj08kxtyKuz+Wsd+i8giEm96V74+12KZSQd2uMn5vxWys0X2+tUMdfjssz7mvdzQh3jq77VWblBZLzR/CQE9XogLcOZNbU5UeE+zF3uQ8fj4oiqJdibnM3JbIkundaBp2HSqsFBQUbjj+kTtLaWkp8+fPZ9iwYURGRvLxxx9zyy23cOzYsX/icv51GM0WPlp/kvqBnnx7RyfUQLnByBeT2+GisZeNNg71YnS7SHzcXejZMIhSgwl/T1e0TuSlgZ6uuGnV3NzMB+/i06IcWTxVvn55SNQ2bSZJ1LVHkKhOOtwBa16QSPBW4+DsgdrTp1rcAm2nyI1+2X2weJosrKqOOVWnLF/GIwJj5PxeITLqtfYl2Pg69HxCunl/fgrbPpS0C48gMTH8cSKgkgJNwVmRBfd8Qua563YTc+YOd0qhp2oRx0pJlqgSijPgh5ulYDJ3NHzTR+LJ71gp+7m4i4+I1fRZrZXiUptJjseoijOgwoFJJIDJJItIBwsXhwQ3gwe2i2lhWCuIGQDTfpV5dcXoWKGSk+lFRF7KWNWJX6WDG/jPqy1DPORvVvLlRJB7h8m4hIKCwiURFeDO17d3xLuK8kGnVfPaqBZEB3tJ4WXiAmg/VVSxDQfCrbOlKBDSTKLFrUUcK2f3SuPEt4588DcbZSx60RTZ//QmKRhUDVmwcu4ApO4RL7px38sI1ZiZcGSJFGus9+aNr8O9m6Bed/G9Gf6BNIRKc5y/2KprkHrdJQzBVE6Fxh0Tmr/2Rv4TGIpkRGzlM1Ks+WmarLmqFnGsVJTWVF3G9Iczu8hV+2OOXy+jUlUpy0O1aIoYVu+dDaO+AN8o+d4PeUu+p1XWMMbogWxOtr3HvRsF0aVBIBtiMxjVJoIhLcJQVRubzSsxcOhMPv9ZdphHFx5gQ2zGeaNjgFAfN2be3hE/D/trD/R05eURzZi51f7vfrHeyAvLjpBXWst6U0FBQeEqcdUVORMnTuS3337Dw8OD8ePH8/LLL9OtW7e/dMwvvviC9957j/T0dNq0acNnn31G586OR0VmzpzJDz/8wNGjYijboUMH3nzzTaf734hUmMwkZBbz9JAmfLnpFLe0r0OFyUKglysrHu7JvuQ8MgrL6Vw/gDBfN/Yl51FhsvD8TU0p1hsxmsw82DeGTzeeqnHsF4c3Y/nBs9zfWI964YP2hQR9kSwIJi6QJCqTQdQ5PZ+UD0mLplQWaSbLguDU+poKkdAW4ovzy4OyILNSmiuPZ590/KIbDpR5+fSjYoy843Nof7uoUQIbw6z+4uHRaIic+/QmMVGeslTk2WoNFKdDZHuRWeckyDGNelmAjPxcikDOiP1VRpiqzoqbjXId/vWlWNNoEPz5mXQBb3oXDKXyulRq6PsfiVhf/bwtVctikfes1TgZf/KNhPIiKZLtmQX5KdJlbDZS4tBr8wXRaGVUbdCrYj6t0V049lXhqmE2W1D/E7P5JqN4TxxdSlnOGVJyX2SIXyoYgqUQWhslWeKP03AQqP/5brSrRkWgm4qUyynkWH0iKsrBRTGGVFC4WNxdtfRtHMzqJ3qTUVCO0WwmwtedIG8dbi4a8AqGr7pBk2HQZKg0XVY9K8qOwIbO1bnnDsCw9xynTNbpBPt/cH5Rp7fI/fSPT0T5uvSemmPZJ1fL77pHkBSOFoyTe36Px50fN6ixFJY0LtLs+eUhLFlxmIe8z+7TOYxsV+dCb9c1hSp1j+0fFWXg4lFz/AlkjeLub/+98AyS9dSa/2AxGdDsneX4JIZiaZBVlMOvj2Ea+y0qkwH1rhn2foGunhgHvEp0ph+vjQqkVaQvRpOZo+cK+OWhHui0aiJ87ZsMeSUGPt0Qz7d/Jp1/7NfDabSM8GHWtI6EVe7fJMyb3x/pyZGzBcSlF9Es3IdIP3ceXniAcwU1m2UHUvLJLzXg7+F6wfdQQUFB4Upy1Qs5Go2GxYsXM2TIEDQa+w7F0aNHadmy5SUdb9GiRTz55JPMmDGDLl268PHHHzNkyBDi4uIICak5ErJ582YmTZpE9+7dcXNz45133mHw4MEcO3aMyMjIv/TarnUMRhNx6UXklBi4o3sDMgvK6ds0hA/WxpFZpMdbp2Vyl7p0jQ4kObuECD935vyRyO7EXCwWiMsIYHSbSD7dnMDQlmF8Oqkd32xNIDWvjCah3jw2oBGYK+jQTE0IpVLcODi/ZurRoYUw4L+SHtHnORmTaj5aihl5iaBCFnIT5knqVcIGWUC0GgcNB8joUdUiDoCLDno/KyNJIGNTzUdVRgdrpYiy7D7IqJIIFbtC5tmDm0pXMCvO3qwPRJ3T9UHp0oF493S+TxZpZXlyXU1ukqJKbR9sXT3ED8gRf3wMU1eA1l0KNTd/LCNoVZOk4laKSeOw9yXJK6S5LDxPrZOvgGiY+gskbpMil5WEDbD1fRnnCnGc4mCHxkUUFArXBDtP5/D2qhMcSS0g1EfHnT0acFfPBlfHcDH9qPzOZB6D0Jac9OqMBRV1U1fAsnliZB7VxfnzD/8oKrc6na78tV4kYZ4qki63kINF/j79w+lbCgrXGy5aNZF+7kQ6Gsv0DIZW42WtUBXr75wz8pJEIdPxTmmG5CXZthkNcm92VOQBuVeXZkPydmmYVC/iWDm+QtYheyqLEHU6yzGbjhCfPxcPWYuc3iRrgL7PQ+JWmLxU/lYUZ6IqSEVlLKduwPWnbLXofDh/pzn8I3R/CDKOi4K3PF8CI0qyoONd8l93f1lLNR8pqZ6/P0VW7zfJL9YTWFvaZuFZ8AyEs/vJLSjmuCGEFlGDCMpLhLJ89PX7Yej8ELFlARxLy2HD8UxS8kro2yiERwc2xM/dhTBf9xrNjqScErsijpWj5wpZsi+V6X0bolGrUKlURPp7EOnvwdCWklK4dF8qidnOR/wvVuysoKCgcCW56m1S60iVtYhTVFTEN998Q+fOnWnTps0lH+/DDz/k3nvv5c4776R58+bMmDEDDw8P5syZ4/T8Dz74IG3btqVp06bMmjULs9nMhg0b/tLruh44nV3CmK/+JDatkBYR3mSXGHh71Qkyi2SxU6Q3MmPraZbsT2Vy17pM+GYHeaUG7u7VgHt6N0BfYWJLfCavjWxB5/oB6CuM/PfmFix9oBuP9G9IlGsRnZO+pt6igajmjoL1/5MO1sBX7S8k97TMPsf0h6Q/xLxXq5M5+O0fQfw6oFJt4hEoXbd+L4raZu93UtSojskoBY1bZ0GvJ+GWr8W88OQaKEgSibB/vZrPO7ZMFh5D3nL8puWelhlvKxYL7Poa5t8KCZtlxv674aIsajXe+ZvfegKU5zneZh3HcveDztOlI+koDjxxi6iYGg4Qs8Gqs+O5p2HHl+ID1OUB++eV50tSRKmT8ytck/x66ByTZ+6i1GBkStd6NAnz5q1Vsdw/dy+GC6W9/FWOr4BZA6T7OvwjGPo2J8JHogLq9JgM3hGw4TXxSrCYaj4/6wScXCvFx2tIwRLiqSIx/3IKOZVF2JyEv/eCFBT+7bj7wcD/yTrB6vnmXx8GvwmRHZw/L3UvNBoo64QRH4N/A9u2E7/JPdcZDQdIg8nVG4rTnO9nMdv8cEKaQ3RfKRi1Gi+F7vh18vjUFXD7zxDeThoyvz5me15YK8oqzAT76Jyc5BqmXg+bkrcoDRoPk9Gn+DXiGTb0LYlh7/IApB2R6Pbh70PzW2D9a+QNeI+P4/xJKaL2lM3gpqIgBshL4uX1mdy2qx5zG33CLx1/4GX97fyZ78fEWXtIyCzm4f4NWfFQT2JCvLBYoE6AJ1qNmrIKE2dyS9mdmMP+5DzKK0x0rOfv8JTzdqaQXeyk0Ae0q+vndFuTUG983G+cKHIFBYXrl3/M7Hjr1q3Mnj2bpUuXEhERwZgxY/jiiy8u6RgGg4F9+/bxwgsvnH9MrVYzcOBAduzYcVHHKC0tpaKigoAAx6axer0evd72x76wsPCSrvFaoVhfwYdrT1JhsjB/VwoDmoUyZ7tj887fDqfxQJ8YHhvQiGBv+RB27FwBm+OyySrWYzLD8bRCNsdl0SDIk6nd6qEvyqVT3mdo45bbDlSSJR/2ej0tqpW4VfJ4aEtRyniHi2lhdlxlikFTSW3a962MFi2eKv+/71vbMVuMEbVNdSpKRH2CSsaMfrzNti39iCRIjPpCxiOqq3kOzIXo/qIw+OMT+21BjR373lgscGi+GC2DKH3CWtm/TisNekNAjIxXOcK/gfjhlOXKKNTvTzreD2RevdfTEpFanGm/7fCPMnqm1kK3hySq3UrKDukQludV+gNdf2NTN8rv4sVw7FwBT/90iG4xgTzQJ+a8AqdT/QA+Wn+Sl34+wrtjL73wfVEcmC9+VvV7ye+EVj6AxOaYCfdSofPylbHEpK0yppgdJxG9ViVXSZb4U/hEygeBa4gwTzW70yqwWCw1vBRqxc1PPKxylUIO/Lt+FxWuAl4h0P0RMSw2VcjfHO8wUb22nij3tup0fxiiukJEO1jxMAx+XUzJizPluZ4hoo6prrbpfJ+EDFSUSWPEO9z5dam1tnCDmz8VX7qkbWJqbCX9sKiMx30vY2AZR8EnXNYNWh20GEOR3oSr15Xpm17R30WtG/R/WeLV+zwP34+wH3VL2CBrjUY6WXuseBjqdscy7F3Kx/7Au1vyWXjgDMeyfenV92VcV0yveQ7/BlIsKskGwBwQzR3dowjzdUelgh0JOayMzaJfyyjMFlgfm8n62Ezu6dWA2LRCbm4jBaKCUgNL95/l7VUnzod2eOu0vHJzc/w9XVl3PMPutKUVRsy1yGqCvHTc3zuar6t55LhoVLw1phVBXjpMJjMZRXpKDSbcXNSEeLvhqv3nx4gVFBT+PVzVvzjp6em8/fbbNGrUiHHjxuHj44Ner+fnn3/m7bffplOnS5PgZ2dnYzKZCA0NtXs8NDSU9PT0izrGc889R0REBAMHDnS4/a233sLX1/f8V1RU1CVd47VCcbmRnafFpC81r4zCsgpKDA466ZWcyy9j84lMHpi3jwfm7WNHQg7vjG1Nmzq+bIrLpEdMEAAtI3zYm5TLTQ006KoWcaqya4YsxkBmqbs/Kh+KXNzEm2XtS7DmPzJu1X6aKGniVklMZdVFVkQ7aDdFumnVObhQFmjNRsDq52puN5tg3X9ln+oYyyF1txgXW+NLrfR41PEiEuzTGeJWSoGpXg8xamw9QTx5bvlaIsndA+GUE9VX72dEQf51r0pzROddIox6WTwVZzjeptGK70697rIIq0rBGfisg7xX5QXOz3GNcqP8Ll4Ik9nCs0sOE+brxr29ou3GqNrV9efung1YvDeVXw6e/ftPfnSpFHEaD4HeT58v4gDE5piI8q68ZahUorbpeKd0ZpffJx5Re2fL6J+pAtredk1441Ql3FNFSQVkl11GBLlPJOTU9AX7N/Jv+V1UuAqUZEvxxWIW1YZ/vcqRaGRUZ/Dropj1qhyVD20Bty8XRa9XCAz/UMafXDxh4W2ihvn2JvG9G/CKbGsxBtrdXrmfh61hE91P7oUR7RxfW8sxouqNbC/jPy7u9kUcK6U5Mn7l6iHNn073wpmdMGYWpuIsDO6hmBwkgv4dXNHfRa1O/PJu/wW2vufYr2jHFzKqZh057fMsqpwE8ksMWCxSTBnVNoL8Ov0oG/yefE+tNOgjCp51r8i/A6JR+dVj8d5UHpy/n4fm7+dsfhnfTO1Acbl9Utixs4W0r+tPoJes2Y6cK+S1347bJa8W6Y08v+wIk7vUrRHQMahZaA2D46r4uLtwf59o5tzRkQ71/Knj784t7SL5/dFetIz0IadYz+w/Ehn6yVYGfriFgR9u4a1VsXZGygoKCgpXmqu2yr755ptp0qQJhw8f5uOPP+bcuXN89tlnV+v0Dnn77bf58ccfWb58OW5ujuX/L7zwAgUFBee/zpw543C/a5mi8grKK8z4VTFmq60TAfJhctspWzrD3uQ8Hpq/n6cGNyHAw5WBzUK4pV0keqNZuhBFKc4PZiiWD0IegTBxIQQ0oFRv5ExuKeVlxaKYKS8QFU15AYz8FBoPFTPCsXPgvs0SE950hPi/nNktC6WqJG0DnygpXjgzRyzOcKxEaTRYFCspOyWJCqQDf9O74BZwvlNUg6guoiACKaIYSsWcccs7MpJiqoBt74NXqCzuJs63T+9xcZeRsfo9YfYAMTZO/lOuxxktxjhOjAD58G31I4pfJ0ogK0GNRY1kMcPKp+39BK4TboTfxYth6f5Ujp0r5K4eDRx293o3CqZbdCD/XXGM/L8zOSNlJyx/AKL7iC+UynZui8XC8RwTdX2qXU9gQ+j5OER1g3MHJbktuKl0ad39/r5r+5uwxo4nFVxOclU4ZCuFHPj3/C4qXEEK02Q084dRkuS46U3H9yWvEBnduX8rPHFMxphi+kvctXW7RyD8dIcoTsvz5T5XlgcLJ0oC5dC3xNNm2X3iSadSyf2y411iSHzzp1jq97KdU6WW8amO94h3T0hz8eZz1owBiPtd1gBtJorKpOvD6LNOs8XYgoScUor1zhtnf4Ur+rtoNMi6y1whSV/OOHdAmmBD/k/G3Ne9jBeljGwTzv+NaUXbKH/6fX6I+4+3Ys/QFZTcuRnzHaukgLbkbinkRbanbMJPjPwhgRPp4qdjtsCG2EyeXHSIugEedqcM8dFxa/tIQn3cyC818PF6x0EXJrOFdccz6NMk+PxjXjotD/driLtL7UMJAZ46+jcNZc60jix/sDtvjmlJ41BvLBb4YUcyb648QWGZEX8PF+7q0YCu0YHEpReRmufAEFpBQUHhCnDVRqtWrVrFo48+yvTp02nUqNHfcsygoCA0Gg0ZGfbqhIyMDMLCwmp97vvvv8/bb7/N+vXrad26tdP9dDodOt11ONtcSXJOCa+uOEZeaQW3dqjDR+tO4qpR4+GioWM9f/Ym1/RNCfbSUVZhomGIF70aBaFSwfZT2Rw9W8jKI2k80DeGjXGZqFQwsXNd1FiwGGuJ5AQZLbp/G3iHcbbQwLurj/D74TTmjqtDN52PJCWZjdLtqtrxuuldGfEoL5RixLD3JUHKK0wKFXErpXAT3UdSm9SX+CMd1kqUNLmnZSEy4iPQF4uyRaWRx5qPrtmF0+rE8Hj189DtYVEJuXqL8mfYe5LQZbGIuubIUpn1j2gLU5bLCJVRL2kY3mFSyCrJkuMmbROJue8Se28eEEPj6D4S6VwdN1+Jb/1xsvxbXySdR5D3q99/YOMbtv13fQ03f1IzLvQa5nr/XbwYjCYzn22Ip3P9ABqHOhghBFQqFbd3q8dTiw/x0bqTvDrq0gziHVKULl4TQY2h+2N2RRyA1CILRQZo4Oug9u/iDo0Hy9c1TqindGUTC8x0qmWiwiG+dWr/IPcv4t/wu6hwBSlKh6V32zcltn8IB36AezbZzIiN5aLg8AqrfQTq3EHHaUoAO7+QxlDDwRDRXtYLLm7S9FhyF/hEkI83fiM+Bn2BpCfpC2W7ixtM+03Gp4zltSc/Wu/3ai1kxWGu24X3T4Yzc1USX0z253BqPvWDPHCpFvLxV7miv4vmiso11QXGUE0Vso4KbQlqFxj2Pu6ePnig5WxqATM2J1BiMLH1VC5bT+WiVsHUjiHc1mIsUY1GUGzW4e4XwjMrU0l3kBKVXlhOcm4pMcFeJGRJpPxdPRpQN0ACJvRGM8k5zosnKbmlNAnzxlunZUDzEB7t34i6gRdIXayCb7V0qqxiPV9vlTHbugEevDG6JV9uPsWXm+WxSD93XhvVgq7RgXjq/jEHCwUFhX8BV02Rs337doqKiujQoQNdunTh888/JzvbidLhInF1daVDhw52RsVW4+LaIs3fffddXn/9dVavXk3Hjh3/0jVcy5zLL2P81zvYGJfFgTP51A/w4L2xrflicnu0WjUvDm9GHX/7FAkfNy3vj2+Dv4cLI9tEsDsxlx0JOdzUMpwvJ7cnPqMIHzctB1LyWbb/LB+sjUOjVlPhGS7KEweY6/agwDWE3bluHEwt5K5vd/PLwXMYzRa+PVRKcZs7ZUffOrLgqt+rcjZdI/GV+UlinLz4dvmwueo5idg+ME8KFlpX+PNz+GGkRF46S4/yCpWCh0+EFEV6PS2pWSufke0RbeHnh8TM78RKkWd/01u6f0Pfka6cV6iMTE1dAYeXiMliTgLMGwOzB8L3w+HgAinULLlDYk1jf4Gjy8BsFtl4RDsxew5qKBHf1iKOld8el4JSlwfkPfGrK8WioW+DoQxunSMFrsCGUghqe5vM5699ybaYje4LBSnyfk76UWb4q46FFKSKP4DCNcXqY+mcyStjdLvaE/T8PVwZ3TaCebtSSKllAXtRmM2w9F7pYvd5zmFx72i2dJMdFnKuI1w1KoLdVSRejiLHpw6UZIoZuYKCwuWTccyxstTFE4rTJUzgi87wdW/4squMZ5fU0iwqrGXM1KiXYy6q9M1L+RN+f0r85ro9IuNWidswmVUSg23Uw6LJMir662PSEInuD3W6SHiDM5qOkDGk70fA2T2UGy3M3ikj/ltPZtM0zOeKqXKuGEVpUFFcmT7Y2fl+dTpB/FqYPw6+GwYLxqP59WGaexTQt0kIx9PsfXvMFvhuTyaDv0thWXoIOwoDyTR5sSkuy8kJYGtcFs3CvNGqVXw6sS2BXq7sT8lj5ZE0sov0vDSsGZ6ujotkTcO8Gd02grVP9ubNW1oRHez1l5IfC8tE5Q7w4vBmPLX4EDtP557ffja/jLu/31vjdSsoKCj83Vy1VXnXrl2ZOXMmaWlp3H///fz4449ERERgNptZt24dRUW1RBPWwpNPPsnMmTP5/vvviY2NZfr06ZSUlHDnnVIcmDp1qp0Z8jvvvMPLL7/MnDlzqF+/Punp6aSnp1NcXPy3vM5riT2JuWQU2vxWWkT6sOVkFvf+sBeT2cLjiw7y1OAmvD+uNQ/0ieb1US2ZNa0jHq5q3l97krdXn+DAmXwOpRbw3po4Plkfz+ODGqNRq1h7TFRQx84VUmowsToZDBMXyUhSVfzrY7z5M15Ylcp9c/eRlFNKXIbtvV4fl0NC9BTME3+U+HB3fwhrCRMXwB1rpEP24232njD5KSKR7nI/nFwNR5aIkS+IueHQt2u+GWqNFEd8o6DnUyKpTvlTCkPlBRKVmR0P4+eJMfL2D6UDZzbBr49Kp7DDHTC+0hg5sDEM+h9s+0B8bayjamYjHF4ki5p2t9vOf2ihKHEcEdjQ/t+F52DBeFEJdX8Mxn4rho2peyAwWowUO98HkxZKMcpkkP2tY15BTUSt1PMpKQQtuVvm/KsS00/SNRSuKebuSKZZuDcNgi7cLRzSMgwfNy2fbnQsKb9ods0QJVjPp+z9C6pwLNtEgJsKP7erEHt+hQn3VHH6cpKrfK3JVcp4lYLCZWM2SxPGEYNelZCDqoEEhhJY97Lc652NhDvzuAHxtirLt91X85JkdLTL/VIAStiI/9pHUB2ej1ntIsqSbo/Ic88dEDP3UxskHlvjCi3H1TyHR6CMk574Ta53zyzUu79mbBsZ5wn0dOW3w+dQ1Ranfi3iGSypnIcXwU1vO26SdZ0uI+urn7NrDqmStuG6eCJB5Nd6CgugUaswmS0EeTlXFoX7ufNQv4Ysvr8rMcFejP1qB2Nn7OCLTafYlZiLl5uWhfd1pX6g/QiWTqumZ8Mgfj2URrivOx6uf10h4+YiBaOGIV6k5JaS5ST96q1VseSVKA0zBQWFK8dVb696enpy1113sX37do4cOcJTTz3F22+/TUhICCNHjrzk402YMIH333+fV155hbZt23Lw4EFWr1593gA5JSWFtDRbvORXX32FwWBg7NixhIeHn/96//33/7bXeK2wMU5SjUJ9dLwyvBl/JuTw22F5L/JLK0jOKeWJRQd5e9UJ/kzIYcaWBD5Ye5L4jGKOnavZSYjLKOJUZjG/H06zM5RLyCqmY/1A/rdLTfyY1VSMX0DFgNcpHL+M/QMW8sBvuXSq7897Y9sQW61DoVGrcFObUG/4rxRMDi2EnV/JgstQCFvfd7x4MxkkBaq6n8ya/0gR487V0GykSH1bjYW714thoUoFdTpC/hkxKKzbVSLLez4p+2nU8MdHNc+XcQxWPSvjTqufh5IMWbAlbXP85h9dan9ttcmTfSJE9VMVi1mKQYYiWSSN+FgWl9Z4VpUK/BqIubK+SCKgdd6i3Ll9mcS3BkbDvu9EJl4VNz/x2rnGjGj/7ZzKLGZXYi4DmzlWtlVHp9UwonUEy/ef40zuZapychNhw6tiEh7ufMT0SLaJ+j7XfxEHIMxLTcJfiSDP/ouFMwWFfzuORnp9IqSpUuQkDnzTGzKS5Qj/BhDayvG23k/D/u/kPtqgj/i+bH4L1rwoTZhK3zxN/BqMvnUxewZjCW0hfi/+9WWMev8cUbEeXgy9nhQVT/2eENZakv1u/1n+Lmx95/xp3Q5+y9RWbqhVkja44uA59MbrrJCTule8hrZ/CBmxcO9G6Hy/jKTHDICJC7F0uk8UTI7WaZmxuJeeo3Go45RMlQraRvmxLzkPsDC+o3Oj5hGtw3lu6WHUahV3fb+HjKJyXh3ZglFtI/l+RxLT5+3nlV+O8dKI5kzqLMep4+/OxxPb8sXmBAa3qN1u4VII9HSla3QAMcFeHD3rPDjiSGoB5RXXmQpLQUHhuuIfHd5s0qQJ7777Lm+99Ra//vorc+bMuazjPPzwwzz88MMOt23evNnu30lJSZd1juuRdlF+DGkRRn6pgbZRfjy0wNbl0le5uWQXG8gulq7BsFbhLN3vXKa8dH8qr41qSdu6/uxIyOGnvWcI8HLl4w3xbIvPZsl+NSsf7c1//wzk+LlC8kpFKbMxLpNb20fSoZ59x39ki0DqHv8GsuJqniz9sER5OiPzuBRnqhLeRvw96nUTY+GKMukieQbZ7xfSXBaNGhd7U9ask7UnOuUlgneomCMHRDvfz2yUL5Dr6TpdOnpW8s+IaXPyHxDcREalTq6Bja9V+ucESOFGrYWve0rhpe8L4O5rO4bWBUKbw5hZUqxRqcAjWEbNQHyJ7lgpo1rWCNa63WHEhzKupXBNsfxAKp46DZ3qB1z0cwY0C+GXg2eZue00r12OV87q56UA2G6q010sFguHMs0MrH9jzPpHeKnYlGLGZLZcmrzexV0ijbPjr9zFKSjc6KjVom49vMj+cd8ox+sAK4XnRCXrCO9QuO1HWP8qHFsm917vMBj4qqhSmwyDHk/A6c2yf8c7ZST63AFZZ/R/GXReuC6eLLHmWz+AgAai3PGNkpHTXx4SBc+B70XR2mW63EfP7hNlb36y/TUZ9XhSxqsjW7Jo7xlCfNzQXW/R1Gf3SUOs/VRI3gEZR2SUu+NdMrrt5i9rj9wEp4fQZ5zk2SFDuH/ePkxm+2LPvb2i+fXQOYa1CufXw2kMbh7GgZR8NlU2IQHUKnhtZEu2xWeBykJBmZGMQj13dK/P0XMF/LQ39fy+B8/kc8/3e/l0Yltu71oPrVqN0WxmaIsw6lVT6vwVfD1ceW9sGz5YG0ewt+OgFIBQH7e/NMKloKCgcCGuiZW5RqNh9OjRjB49+p++lBsGi8VCu7r+3D5nF4VlRlY91ovsIpv8c3dSLj0aBvJHlWSqizsubDyRyZebTtGvaQhfTm6Pv6cr83bKIuauHvWZtT3R4XGX7j/LXT0aoNOq0RulIz6hhQfuvy10fLKidPCrJyNGjvCvJ2kHVen3ki0y2RpX6gitK3gF13zcxU0+sFWUOX6eT6QoeVRqkVM7Q6WSBK3+L4nBokkv6Rk+4WKmOHuQvTeOxhWmLIWH90khKesEHJhrW3geXgQJG6UjVr0I4+YjXzVeow7qdhE/n/J8uWb3APBwPD6j8M9hNltYvv8sXRsE4qK5+MW+TqthUPMwFu85w+MDGxPg6XrhJ1k5tV7GFfo8Lz/zTkgptJCvtxDjd519CHFCuJeaCrMYONfzvcRFtm+k/G4qKChcPkGNZJz56FLbY8WZoqxxhkeA3Ced4VtHlKv9X5L7rauXGCTnp0DGLHuz/72zodnNUqDZ8YUka258Xe7vQY1lDLs4XTzx0g5K48fqw2OxwKmNkPQHjJkpjRJHqDUEBwSwZW8m62MzmTGlg11y6HVBo4EyEp52EOp2ljVZfoo0m/zqShEspJl8X5x47um96rD5ZAbf39mZuTuTOHq2kFAfN+7oUZ/m4T6UGYxkF5Xzw45kTqYX89TgxtzVsz57k/LwdtPSo2EQOcV60gvUvDaqJUdSC1CpoE/jYO763nGS1uu/x/L80KY8t/Qwt3aow8g2EX/7WxMV4MELw5qRW2Jg9vbTmB0Ikh7oE0Owt2IKr6CgcOW4MVbmCjVILyznnh/2UlgmqhAXtYoO9W0f4OfvTOG+XtE0qZaMc/xcAeM61nF63KEtw9h2Mut8LOTrv8fi7qIhNa8MX3cXRrWLZOURJ9JoYH1sJh9PbHu+M6XT4LxocnSJKFkcoVJBm0mSWgUiyx77rXTi3C9e0VADNz9of6fjbV4houApy4N6PeXfIc0d79t4mCwiE7eJEfLCSTCzH8weIuNZLtW6QyaDJE5ZLLK4XXq3rYhjpSRLtpkvUarrFSwL58AYpYhzjXIwNZ9zBeV0bxh04Z2rMbhFKGYLzN+ZfOGdrZhNsOYlGT2s16P2a8uSn7cbpZATUZlclZB/GZJ337pKIUdB4a/iGSxedlOWS6BB/V4y3txwAOgcNCVA/OK8LzAeo/OUBk9QY1kTVJRBZqwYG1cn9ldJu2o9DnZ+KUWckZ+JwfHCCZX/nSjjVG5+NYvdRr2sBZwocyua3sKnO/PZcCKTqd3qERN88SlJ1wzBzcAzFE5vgQWV78ni26XoZTGBqQJLRiwVrW5z/HyfSNyCo4kJ9mZHQg7Tutfn00lt6d80GIvFwr3f7xXTYJWaB3rHsC42g1u/+pOfD5xlaIswmoV5sz85j/fXnuTjDfHM3pZIu7r+DGgaQmpemVPLpKwiPW4uGoxmC4v2nGF9bAb7k3MxO6q2/AVCfdyoH+TJV1M61FBbjWkXydAWoahqSzqrhfxSAwmZxfx++Bzb4rM4m1eG0XQZI8EKCgo3NNeEIkfh76eovIL/3NSU5NxSftqbilaj4vGBjdkWn43JbKFIb+SxRQd5enATgrxcKSw3Euylo6CsAg9XDS0ifGr45DQO9aJhiBcfrLV5RJzKLCazSM9nE9tRbjRzKrO41qTK5NwS9EYjn01qR16pARcPC5aorqjO7Ky5c0m2LMaGfQBr/yMLJ5BRqWHvSTz4HaskItNkgEOLoO/zlxepXZQhZsJ7ZolpYUEqnFhh2+5XF4Z/KIZ+A16tTMDyFlPmH2+zjS6BfDAe9p54jyRusT9PfjL8PF3GpH55yH6bvlBGt6o/pyqxv0KHO+3HwRSue1YfTcfX3YWmTiLHa8PHzYVejYL4fkcS9/eJwfVi5PtHlkBWrPxuXWCheSDDRKiHCm/XG0MiHuiuwl0Lp/LN9K93iU/2qwsnfhVVnYtzSb2CgsIF8AqBhv2hXlcpLOu8xQh52q/ikWcNOFCpZPSz7W0SWnAxFGdC2iHIOQ1xvzvf79ACGb8y6qHPs5JmZQ1OsJKyAza9KffdnV/ab9vyDoz6QvzzqoxcmmMGUNTrZZqe07CiUyOMZgtl16NXSlGahD9U9wLMT4Zl98LEBWSlpZDZZDpNizNxObXSto9/A/TjFzLpx+TzARdfbD5FHX933hvbGq1aTWJOCX8mZLM+NpOoAHeev6kp//d7LCsOnWNa9/qsOJTGor1nzh8yNa+Mtccz+Ob2DpRcIAFMq7HdrxbtOUOXBgFkFpUT5vv3hjwUlxs5mV7I57e1I7NQT4nBRMMQLwpKDVwwtt0JWUXlvLkyluUHzp1/zEun5ZvbO9CxgT+uf3OEvYKCwvWLUsi5wcgrMZCSW0pBWQUatYpO9QPo3zSE/FIDqfnlfDKxLZ9tOEVcRhEFZRWsO57BPT3r0yLCh1Ff/EFuSQXeOi1vjWlFal4Zq4+lY7FYGNwijAZBnjy/9EiNc/6ZkEOYjw690cyBlHyGtghncZWbb1X6Ng7m/1bGUl5hZnrfGJkvdnsbZg20ecpYieoinhTtJkOjQaJkUalFQu3iKQuuPTNlbKjVeOj3ghR+LpWiDPj5QUhYL/9O3i7+NLf/LF0njU7OkbBRpNQBMVLEAZmjn/qzLBxLsqVj6BkM+mJRFDmiIFUUOS4etrhwK/qi2ke23Hwvr1ClcM1isVhYeSSNjvX8UV/mPP3QlmFsOJHJ70fOcUs754o6AExGMfuM6ir+TBdgb7qRxgE3hhoHQKVSEemlJj7vMrqbfnXFiDz7ZK3m0AoKChdJVXWqWi0+d/duhuI0uR/6RonHnZuv00PYUZIloQdHfoKeT9Q0+69KeSEWjSsqjasoeasXcayc+BXLXWtQVS/kFJ6jKO0UKQPnUc+tFK0+D5NnGIUaPwrxZv7uo7w0vBm7EnPo37SWUe9rFVdPiP3F8baCVCxl+SSo63Pn/ESe7v00Qzo9h58xC1fvQAo0AdyxOMUupRSkGPPJhnheHNYMNxc1STmlhPm6cTAln/t7x/D9nZ0wmiy4atX4uLugVaswVlHS6I1mPlh3kndubY2Hq4ZSQ82CTus6vpzMsCXhlhpMmMxQYXKuyCkoFa/I+MxifNy01Av0JNRHh/YCo84HUvL4YJ0U8YK9dOhc1KQXlGM0W/j2jk70u8Tvu9ls4ecD5+yKOADFeiN3fLuHdU/2pl7gdajuUlBQuCIohZwbiFKDkTN5pbz48xGOnrUtXno2DOL10S15ZskR9EYzU7rWIyrAHbVKxe7EXN5aFcdtXeqSW1IBQJHeyMMLD9A83IfejYMY1CyUt1ad4L3kPIfnDfbW8fXW04zrUIfCMgMTOkWxLT6LtAJ7Y8JhrcJoVceXpdO7E+LtZlMOBDcT75e1r0DSVlmwdb5fDBG9KxN8/OvJV1U8A8V/BnNlKtRlkn7YVsQBMFVISsP2D2Hg69DtQTFZbDTEZiRcFa9Q+apKeaGkaGQedzw7Xpwhr7N6ISe4iRgLxq91fK3dHnYcAapw3XIyo5jUvDImd7lUeYiNOv4etI70Zc72JEa3jaxdzn10iXxg6fH4BY9bbLBwPMfMXa2uM2+HCxDupeJk7mV0yK3+VFknlEKOgsKVQKUC3wj5uhxyT0sRB8TMuH4v+zjzKuhjhlKiCSSgQZ/aQw4sZowqF8pvnon3n+/KOiAwhuzoW9hc3pBgVQCPLEkjvbCCsookAG5uHcHUbvXILTHQto4/GpWK3BI9AZ7XkWeKSmU/yq1Syzi5xkX+BpZk8cpGC3qjmf/bmM7/Ac8NbUJmQjk3tfImNq3I4WF3ns6lxGDiyUGN6VQ/gPIKE77uLjy04ACJ2SXn9xvaMowPxrfhycWH7IySD6cWUGow8vGEtjw4f79docffw4UnBzXm2SWHzz+m06qJCnCnqLyCY2cL8Pd0JdRbh6aySJNVpOetlbEsO2AL+vDWaZk5rSPt6/k5VcAUllUwa7ut+Fc9hnzmttN0ahCAl+7i16eZRXpmbHFsHm0wmdl6MovbuylrQAUFBUEp5NxAZBaW88KyIzVGorafyub/fo9lUPNQvtycwJsrY+22Nw/3IafE/gYEcDytkONphaTkltIy0pe9Dgo5Hq4aIv3cSM0r47s/k3hpRHOeW3qY10eLKd22+CzcXTXc3aMBrer4OTZ+c3GTLtz478V0UKWWueyLkY+q1fwlqyejHvbMdr593xxoM8FWULoQZflSpIlfBw0HyqjXmd0SaV51oNu3jpgmV6XDnaLmcfOFFrfCsaX229vdLu9TdYozJEY6+U8pKNXrIcogZfTjumB9bAZuLmqahzvxhrhIhrQM4701cexPyaNDPSc+UWYzbPtA1G6BMRc85sFME2YLNLmBFDkAdbzVrMiowGKxXJqHgaunqASrjlIqKChcOxyYb/v/xC3S/Di0QBSzVfEIIKPhWEryivBrNxW1Zy3eelod5Rpv5mQ04u5bfsAtdRuqihJM3pF0DPFj8oIjnM23b1ytOHQOHzctw1qF88bvx/n8tnaYalGEXJNo3WzK4XZToPloSbIylkPvpzH7N+RcXtL53V00KpqE+bA+NpP+zWpfM5UZTIT5uDHxm528NaYVTyw6RHqh/Xu4+mg6QV46vp7SnqxiA4dT8/nl4DlKDSayigz8uCeFWdM6cvBMPqezS2gR7kN0sBev/xZLZpVwj3Ed6/DroXPM3CZFF193F/43sgUDm4bgqdPy84GzdkUckIbmtDm7WfdEH+o6Sbwyms0UlVc4fY1F5cZL9rUxmc3klDg2jgY4XaXQpaCgoKAUcm4gCsqMNYo4VjacyOCunvX5cnPNSn9STgmdnH3wA+LSi3iwb0Pi0ovYcdpWfPDWaXl3XGu+3CTHzCutwE2rIa2gnHu+30uHev7c0aM+fRsH4+N+ER19d7+r7/1iscj4lNPtJuAiF1+lufDnZ6LkqUqbiTDodVj7kvw7oh341YfAhvKB0Dscej0liyTr6x/2LnR/CI4ul8JWyzHgF1Vz7KrwHCy6Hc7utT2m1sKEuRDdXynmXAdsiM2gVaTvxXnb1ELbKD/Cfd2Ysz3JeSEn7ncZC7rpvYs65q40I96uEtl9IxHppaKkAtJKLJf+2vzrQfrRK3NhCgoKf42q93OLRfxrxsyE/XPF3wrQNx5FTqcncVdbqLtihDRVBr0OdTqJV1419O3uRq/x5IGAzehmP32+KRMKmBoN5Z0hrzBlUVKN5y3Zn8ptXepy9FwhJQYTEX5/XwT21cCYfw6XLg+AoUhGz+aPtduubjaKz0Y+w91LU3B30fDWmFbM2Z5I38bBeOqcN+J83LX4erjw0fqT6FzUqFSqGkUcK0v2naFP4yDeWhVL95ggZkzpwNrj6Rw8k8eu07l0jwlkeKtwFuxKpmM9f55depiELFuxY0iLUKZ0rceIT7eff6ygrIInFh3kx/u6Uj/Qw6kCRm80sy0+i8mBjtWyPm4uDGoW6lR5NLRFKN5ulzYKr9NqaBjiJX6TDujS4C+EeSgoKNxwKIWcG4i8UudVfItF5KXV541BuhOhvm50qufPHgeqm+eGNgWLhbfGtCK/zMCu07n4e7ripdMyY3MCh1JFkuzuosFSRXWSkltK2yj/iyvi/FO4uEH7ac5HmVpNAI+LTBLKjK1ZxAE49CPc/KlEqYc0heEfyNz/1BUSk6rWipKmqjLAM0i+Ijs4P59RD398Yl/EAfEaWnQ7PLxXPHwUrlkKSis4eCafu3s6Tj65FNQqFUNahDF3RzJn88uI9HNg6vjHJ5JUFdLsoo65PdVI80AN6stM3rhWifKRollcrokIr0ssoPnXr2n+qaCgcG3QZhIcmGf7d+5pCSRoeSvmyctIsYTw3cEieuWYGbBjqk0Zu/VduHUW7J0j6wGLBbQ6zB3vobzDg5jy09CtfqrG6TTxq2kR2Zt2UW04cMZ+PKu8wkx+mSg2Cssq/nKx/mqj9q8LhnxZn3x/c43tqthf6N5gADOmDEalUjF7eyKnMot5oE8MZrOFQc1DWXc8o8bz7u8dgwoZkWoQ5ElavpPkUuQ9rDBZKCwzsvpoOpvjMlk6vTuZheWMahvJ0n1ncHNR0yTMh4cWHGB63xgi/NwpNRjxcXNB56LmrZUnaqx7Ad5dc4KPxre9bAWMVqNmXMco5u5MJq/UXpkT6OnKqLaRaC7R9y7IW8d/hjXjru9qFhRDfXS0ruN3ScdTUFC4sbm+7ioKTskoLCfA03nBRK0CrVrNl5Pb06aOmAZq1Cpu71qXWdM6UmE08cSgxkztVg9PVw0atYrGoV58NKEt+5LzOHimgGeXHkKFithzBby1MpYH5+8/X8QBmNApiti0AlpF+PJAnxh+vK8r4T7XgSIksgNEdqr5uG8d6DANNBdR7zSUwo7PnW8/tFAKN2NmShEHJBbct46MQV3OB+WSTNj/g+NtZmPt6VcK1wTbT2VjtnD+d/Kv0qdxMG6uar7/M6nmxpRd0m1ucctFHavIYOFwlpmWQTfebSKoMrkqLvcyDI/9G0BRuijwFBQUri2CGkPDQfaPVZRB3Cr0nhEM+TaR7/Zk0jbAIEUeK/oi+OlOiROfuADLPRswP7ib7M7PkmrwJjjBSXgB4H9wBne3Fd+SFhE+DG0ZRvu6fnjrtGRXjvj4e1zDDS0nqMrz4fgvEsHuBPfdn2MozODB+fvxdXdh7t2defmXo5zOLmVSpyge7BuDd6VHTISvG6+PasFNLcOYsSWBSD93sor0RAV42L1vVfHSae38ccorzLy3Jo6t8dkcO1fA/F1n2HU6l+GtwvlsUjvWHkvnlV+Osmx/Kt5uWuIzithyMsvhtZ9ML8ZisdQaDd+pfu0KmKgAD5Y/2INRbSNw0ahw1agZ0y6SZQ92p07A5SmwOtbz55OJbQnysv3MdI0O4Mf7uhHhqEGjoKDwr0VR5NwglOiN5JUa6Fzfn91JNVU1N7eJYM3RNJYfPMekznV5dEAjwn3dOJxawNdbThPqo2NQ81BGtonglnaRlBpMeOm0bIrLZOa203w2qT27E/OYNHMnS6d3Z3dSHnnYOhD39mrAre3rkF9WQbu6AXjqtHyzNYG7e0TTOMyb3BI92cUGMgvLCfDSEeylc+yX80/gEy6jSPHrYO8sUbq0ngCtxsk408VgMkhhxRmlOeKv4eZrG+f6KwbNIOlD1c2Sq1KY/teOr3DF2XIykzr+7gR6/T2/C24uGvo3CWHBrhQe6d/QXta980spHNZxULR0wK5zRkwWaBl840WdqlUqorzVnMi5jEKOVeWWfgSi+/y9F6agoPDX8AqBUZ9D4lbY9ZUkSDYbBe1vp8w1nJFtSsguNuDt6QnhbSHtoO25FaWw8yvY+RUFk9cQq/eiSF9OsxAvVIWpzs9ZmkujYHdmT+vI8bRC4jOK6d4wiJdGNGftsXQGNgvBRXP9qRqNZYW46rwvuLbpUteHTyeGsicph3P5ZSRml3DsbCH9mwUztVs9RrWNwFypCvd01ZBdYmBa9/qYzRDmq8NktjC8VTgn0ovo3jCIR/o3YsaWBHYl5nJ717qsO2a/ltkWn83Y9nVwc9VQbjQxb1cKA5uH0qlBAF9N6UBZhQkPVw3ebi5si892cuEQFeDO3qQ87usdw3NLD9fYHuKto/VFNFnqB3ny1i2teP6mpmABP08X3F0uf33n4+7Cza0j6NwgoFLJpcHfwwW/67AYqKCgcGVRCjk3CDL6oOI/w5rx8fp4Nld2INQqSU94qG9DJs3cSU6JgRlbEpgxpQP3/rCPs1UkrTO3JfLKiObsTc5l5ZF01CoY1TaS98baDHZLDSbm70zmnVtbUaQ3UVBWQYd6/pzNK2P8NzsoLJMIca1axR096hOXUYSnTsNTPx1i52lbB7tJqDczp3ag7rUSo+gTDh2mQtPhEi/sEQDqC3yArSiT2PG8RND5QsxASN3reN/oflK4OXcQ9n0nEaktboG63cA38vKu2dUTgptKeoQjGvS6vOMqXBUsFgtbT2bX6ED+VYa2DGfV0XQW7k7hvt6Vhsb5ZyD2V0mDU12cwmZDipFwTxVhnjeeIgdkvCr2cpKrvCNAq5O0O6WQo6Bw7eEdBq3HS+CA2QTu/qDREqAv5p1eWlQHf0S1ORma3SyBBGtfgpxTtudrXCl38WPGxtMk5ZTw9e3tMTUejva44yhuU1Q3vH39efzLgxTpjecfn7n1NF/c1p6BTUPRatQYjCZctddPYdzkWxd8IiUd9OQax/vU7UlyiZaHFuxnSte65wsnXm5aPloXz9Su9fD3dOHI2UKGtwzj6LlC/vfLMRoEe9I03IfejYKYPm+/3fum06r5cHwbejUKolm4D22iLPi4uzBvV8r57S5aNck5JVgsYDJbzjsZ+ri74ONua2CMbhfBl5tPOYwef6BPDB+tO8mAZqG8PKIZX2xKILdyzGpA02D+e3OLi1bAeOi0eFxCOtWFUKtVhPu6E+6rKHAUFBScc2Ou0P+FBHi6EODhQmJOKU3DvZk1rSNfTm7PN1M74u/pSkJ2MeM71gFgTLtI5u9KtiviWHnj9+OM6yAqFLMFlh84S0JWMWkFtn33peSxOymPx388yDurYynVG7lv7t7zRRwAo9nCrG2JGIxmZmw+bVfEAYjLKOLeH/aRVVQzLesfxTNQRp4uVMQpL5SI0y86wQ+jYFY/iOoMbn4193X1hM73wp5Z8E0f2PctnPgNlt4N3w2HnAT5oK0vhYKzkkBVmHbha/UKgaHvON4W2uqiUokU/jkSskpILyyn1d80VmUlwNOVHg2D+GZrIuUVlYWKPbPAxR1i+l3UMSwWCxuSjbQLvX4+dFwqdX1UJOSbMVxqkoxaA/7RUpRVULiRMRmhIFXuSUUXcU+60uiLID8F8pIlIfJCeATI/VyjlcbLid9Qf90D1c4v4cTvsPF1+OUhuOkduyCBkg4PUKgNoHmED8k5pSRmlVAU2kkUjdVRaynv/SLTf4q3K0aAmOU+u/Qwbq5q9BVGCsqcJxxdi1hcfbF4hYFPhBTHqqNxIaP94xzNMvJQ32iGt4ogOacEtUrGgw6l5rMlPosf95zhz1PZVFgs/LT3DK/f0pLGod60jfLjlV+OOXzfXv7lGO3q+nPPD3t5YN4+6vh7MKi5JGGNbhtJhI8bB8/kM7h5KFO71cPP3bGpcKSfO3Pu6ISPu63IolWreGpQY7x1WkoNJmZvT2RLXBbf3dmJXx/uwZrHezGuYxS7EnOJTSus1X9SQUFB4Z9EUeTcAOQW6yksq8DdVcN/lh2hrMIEW07b7bPqaBoL7unKvJ0pIl1deMDhscwWiE0rJCbYi4Qscc2ftzOZ10e3PL9PoKeOonIjBpOZ9mF+rI/NcNjtAPhm62lGtY1wuC0uo4isYv21M2J1KeScghWP2P5tscDaF2HsbNjxFZzeII/V7wk3vSv7bHqj5nHyEmHrexLtuWeWFHnKC8CvLgz4L8T0l8WoM+p0hCnLYfVzkkak1UHrSdDnWccLL4Vrhu3xWbhoVDQN+2ux444Y1SaCbfFZ/LT3DLd3DIP930PMACnmXASHs8xkllpofwMXcur7qDGaIT7PTIugS3ydgTFwzvHfUAWFG4KiDFGP7vwSyvOliDHgv6Jyqe2edKXIPiXKmfg1opqt30sKMEFNLs7HrjhDijaWamuV0hzY/pGEHhz4gYJOj3PUfxA+uNK3cTA9GwYR5KXj8ZVn+Gjccrz/eBOXuBWi9IloR3rP10kzhXModZ/D0+aWGNAbzYD6ko1v/2nczEWoVj8LPnVg1JewZ6YocyxmLJEd0A96mxfWlXAyO493x7bmj1NZlOhN/G9kCxbvPYPFAkFeOo6nFVKkN3LsbAH39o4mv7SCFpG+RPi6OTUTzi0xUFhegQrJDf1o/Uk+v609pzKLuadXA05lFuPhqkWrNhPq40ZWsZ4Q75qejK5aDd2iA1n1WG8yCsoprzAR7K1jf0oevx5K4+khTbBYwM/Dhfm7kukRE8Tzy45QarCpNYe0COWN0a2uz7WqgoLCDY1SyLnOSc0t4eCZAjKLy2kS6iNFHMDHTYu/pytZRXpKDSYyCvUUlFcwa1oHXLQaO/O46hTrjbi52MRaheVGtGrbv29pF8k7q2WcJ8Tb7XzUo0atolWkLy4aFSfSiijSGzmTV0qIj/ObX27xNabIcYTJKN3I7HgozRZj5O0f1dwvMxaW3gMD/gfD35MRFjc/iRTf9Jbz4x9bLjLwPz62PZafIoqdER9Bu2mgcfJBU+cFDfvDHb+DoRjULuAZrMSOXwdsP5VNoxBv3Fz+/mJJuJ873WOC+GzjKca57MCtLA+aDLvo5/+aUIGvDpoG3Liizbo+alTAsWzTZRRyGoqqrrwQ3P7+QpyCwj9KWR6s+Q8crWLwW5AKy+6FYe9DhzsdF08MJTJunH5Eii1hrUU5qvP6a9eTnwJzBtkbjCdtg1kD4YFt8vt4IVL3SQiAI5K2Yxn6DrFR4zlV5k2otztTZu+ioMyIh6uGmVM7cii1gD9yoiiv+zwtWz2NyWzkcJaFL1bk8eJw+/WUp6uGIG8deSUGCsuN6CvMbDqRyeMDG/+FN+Hqozp3QApW+cnw0zTxDhz3nWzLigN3f7YknARkvH90u0gGNgtlzbEM9iXnoVJBj4ZBfPdnEkNbhNImyo8Xlh1hT6WP45eT29d6fn2FGa1ajcFkriyGWfhqcjvm70rm98PplFYYKSwzsvzAWTrXD+Cz29oR6iBgQ6tRE+nnjoerhllbT/PFZlvc+PKDZ5nQqQ71Az0Z0iKcB+buw2Cy905bcyyDlpG+PNgnBo3mxr0nKigoXH8ohZzrmPIKI0fOFfLwwgPMmNIei8VCVIA7TwxsjFqlIr2gnDoB7hSVG3l/TRwVRjO5pRWYzQaahnlzIr3I4XHbRPnx7R9J5//t7+FCuVEKRFO61CWtoJzMIj0qFQxqHkpSTgm+7i4MaRHGnqRcyitMTOten8xCPauPppGa5zxaMuRaT7UyGSXpZ8E4kXUD9Hoa8pIc71+WB3u+kdl7T5tU+/xzHZ5D73z7hteg0WDHku6qeIUAIbXvo3DNYDSZ2Xk6l5taXjnV1Jj2kTz90yG+33SQ+yPaXbQXk8ls4ddTFXQJ1153HeRLwU2rItJbxdFsE+Mv9cnWD45phxQvKoUbj5Is+yJOVTa+AY2H1gwCKC+AQ4tgzfPy4R+kmdH3P9Dp7stX8ZjNcHSZ45S4ilLY8SUMfRO0F1hLVDiPkQagJBuzV1tiT6fx6I+Hzj9cajARn1HEF7e1IzG7lNPZJgwqb/w8XPlqTyxn88swWyx467ToXNQ8NbgJ3jotqXllhPm6YbZYqOPvzsLdZ5jStR5h15PnSXmVOHV9kaiG98w6/5Brk+GAmAabzRZSc8uoMJsJ9XXj3VtbExngzqrD5zCZLXSsF8Brvx4/X8QBEUd567Q1RqtAxp983F3siioalYqyCjM9GgZTx98DbzctnjotC3edYVDzUE5lFrPlZBYh3joi/NzxdXPB39PlvC9RSk6pXRHHSkywNzsSJEGyehHHyqxtiYztUOeyPGuKyyvILjGQW2zA3VVDoJerQ/WQgoKCwqWiFHKuY9IL9Ly7Og6NWkWAuwv+Xjr+b3Qrnl1ymPTC8vP7xQR78snEtuiNJqbP20/dAA+eG9qURxbup7owp3ejIBKzSs4re0AM4TQqWPFwD8orTCzdd5Z7ezWge0wQyw+c5c4e9dkUl8md3+05/5w5fyTRLTqQ/xvTiiV7Hac9/D975x0fVbW14edMn/ReSSGE3nuXIooFFRGwo9jF3vV6bffqp157xw6CBRQrICKI9N57CyEJIb0n02e+PxYhhCQISgvs5/eLkjll9pnMPmfvd6/1rl4pYbXKK56WlO2DyZdLfn01xXsguo2YndZHXFcwH2bi3HoYLHuv/v1TBolYVB+2YhlM/ZWQo2hUbNhXSoXDTbv44+uPcyixwVYGN7Xyblo3RnWM4GinUQuy3ORW+ejf5MxNq6omKUjHxvy/YXgcnCBpavtWKSFHceZRsKvhbfaSAxP8w4Scgl3w6yO1X/N5JaU4ocffNwZ3VsCOXxvenjZP2hPwFxPj+G4NbwtPxamzUuX08MFhaemtYgKJD7Vy6xera6XbRAaaeW1URx6aup4pKzN58LwWpET685/pWw5GKQPEBluYMLY7Yf5GGo6DPk2Jat3wtogWaEYLTUKt3NIvBatJz11frSG3rCbKukmolY/HdGPhrkKSIvxYML12BakpKzO4a1AqL82qW7Dhhj7JzNiQffB3k15HfKiVudvysLs8lNvdzNy4H6/Xx6Sbe/LA1HVs3V+zIBYRYOLVUR1xe3z0SpFKql+tyKj3Uox6DavJQE6pvd7tAKU2F+5j9VMD8ssdvDV3B18tzzg43k6J8OfD67vSPDrwmM+nUCgUh6JiBBspBRUOHG4Pzw9vw0939eW3rbkAPPXTploiDoip6ltzdrKnUEpVZxRV8dO6fXx4fVd6pYRh0uuIDjLz+AWtuKFPMq/9vh2QygBj+ybT4sDDpszmxuHycNegFLZkl3PLF6v4bXMOZTYXb8+tO/BbmlbI/O353H5OCld3T8BwYHVfInmieGN0J8L8T/Oc48zltUUckJSK9qNAX4+5ns4Ave+quzoYngpN6xnIGiyyYrl2csNtMJzmn5HimFmyqwCrUU+zyH+YcvAXjLSsxIuOl7LaH/UxkzY7aRqs0SzkzH88NA3WsbXQ26DHV4Po9BDeouEqdQpFY8YacuTthz/fXDZY+m7D+y96XdIQ/w56I/gfIdrULwz0R7EgFBiLr9O1dV/XdDDwCfZ5w9hfz0T+9gHNeHxabc8UkAn6q7O3c0OfZHbnSbnxd/7YVUvEAdhfauf2Sat54fL2hPjVb8h72mIvg3ZX1H1d08HAx3FrRl6+ogMtYwJ4bNqGWiIOQFaxjYe/Xc9LIzrgb6q7brxgZwGFlU5eHdWBNrFBGPUazSL9efbStkQGmJm2Zt/BfR+9oCUA2cU2Zm/OZXtOOY8MbcnLV3TkpV+31RJxAAoqnDw2bQOlNhdpBVLdqiGz6RV7imgSaqV1bMNpss0i/WtZDhwNLo+Xr1dkMHlZRq1F07SCSq75ZDn76yk4AmB3ecgsqmJ7ThmZRVU1RQsUCoXiMFRETiPD5faSXlhJemEVhRUO2sQFcdVHy6h0uhnYMoq9B8Saw1m5t5jbB9ZUMZq3PY8uiaE8cWFr9JpUmXr3DxFj3r+2K063F5NBY8aGHO6YvJoZ9/bD54W8cjt7Cqp48PzmzNkSTEZRJbM25zTY3s8Xp3NpxzieGtaGOwY2o9zuxt9sICLARKClEQxq6kuhcjsknPvyj2DOs5I/DmJQfNn7EJpc95iAKBjxkfjhLB8vFTea9oeuN4LjCAPc5P7gF/GPL0NxerF4VyGtYwNPbOqSx0Hw3tlcGWXmsx3NGd7CTZ/4I9/ytxd5+CPDw20dTWjamZtWVU1qqA67B7YXe2l3rD45kS0gbb7kB5wFn5XiLCIkUQSS+tKZkvrWqvAEyDOxtP5oBwDKssFtB/6Gn5TRCr3Gwdaf69/e5z4pL/4X+Kwh5HZ7mKgmPdAtfx/Kc6Ssdu+7cAQmcdc32Tx4fqtax5j0Okx6HYWV9Vct2pBVyn8va0d4gImcUhur9hbXu196YRVmgx6rsZENuUMSpGBDXGdY84X4H8V3kSqcFYXkVGn4fD7K7d46AlY1m7PLKLO7cHo8aFpdr+mPF6aRGObHpzdIxJTH68Nk0PHh/N0Ho6FGdU0gJcKfy99fTOUhgtqS3YVMua0X83fm1/veuWUOzAYdny5M48URHRjeKZ5Zm+qOV3/fksvU23uTU2onPsRab0XXJy5qTeQxpkPllzv4eEFag9t25JUTe1h587wyO+/N28U3KzNxuL2YDTqu6p7AXYNST38rAoVCcdJpZE8Vxd6iSm6ftJp9JTb+fXFrfli7D4/Xh9mg+8tS3k635P7qNHhjdCemrdnHS7O20SY2iEGtopizLY852/LqPdbm8rJiTxFvzd1xsMx4/9Rwnr6kLW/M2dHge5ZUOfH4fPiZDSSZG+HXrUn3+l/fPVf+f8MvYvCITwa3R6oUFRgDPe+AtiMkt3/LjzB1jETrXPoOfH+bvF5NSKK8/lero4pGhd3lYfXeYq7snvDXO/8T9i4FZyXn9ohnmUfHQ/NszBzpT6il4VXFN1Y5iPLTzoq0KpCIHL0G6/I8xy7kRLSCjd+KEWto0olpoEJxKgiMg2umwheXHXi+HSC4CVz2LvgdJpyY/CGxb8MRak16gPkfpJFEtoR+D8Gi12q/3ulaSOp9VKfw+eCFBSX4mTrx8KWT8de5sXt1LMhwMndlBf+6uA0Ol5eUCP+DlZTMBh2VzgYMkg+QXWLjie83/qVxb5ndRXGlk1D/0zyd/BC8bhd6TQcbpkhlTWuoFHVY+Rm+859n7oZ8WjRNxPEXESOVDjcbs8o4v00Mv9Wz8JcUZmVHbjl3fSWVAFtEBzD55p7YXR4mL9+L1ajjmZ831RJxqimocNYRhw6luMrF/lI7NpeHTgnBtIgOYEduRa19TAYd4QFGooLMvHtNZ16bvYPFuwvw+SSF7t8Xt6ZH8rF7PNldnnr9f6rZnVfJgEP8r8vtLl76dRvfr62JRHK4vUxcupcKh5vnLm1HgKURjqMVCsUJQ90RGhH5ZXbu+WotF7SLoXNiCOszS9GAN6/qxLacckL9Gh4gmPS6g6lN57WJZuXeYuZtF9Ems6iKVjEND7Ligi3YnW7+O31LrdcX7irkP79s5oquCczcWH9UTt/UCALNjSDypiEiW0FYChTVs6rS5fpjn8BpGgRGy787XSurm2l/iqny7QukpHFRmpQVj2oDQfWXblc0XlbvLcbp8dI27gRXO9o5G8KaoguI4M5OXv610M7dc2x8fqEfJn3dCJJ5GS5m7XEzrrPp4L3iTMek10gO0rE218N1bY7x4KgDq/cZy5SQoziz0OkkWuXOpZC1Agp3Q3xXiGoLwfU8k/RG6HqDGP0fnoqsN0LfeyWy5u/iFybn6DAadv0uz8vmQyAo/qhNlHU6jRFdmzD285VMqUdvMhv09G0ezutXdmLc5NVkl9opd7iJDDDXG0kCEGA2kBjmx+MXtsKo1zDqtQbTNCMDzBQ2MiHHbQlD73HB8A/Ei6goHVpcCK0vIS8/j8/WeHi3lQENWSCsrxiqUa8R6mfi44V7+L8R7dBp8NvmHLwHAhnPbRXFdb2SuHPymoPH7MitYEduOf2aRzK2b1NyS+0sTasnOuwAfiZ9ndS3amJDLMSHWvEz67EazUwY24PJy/by1YoMbE4P57aO4sHzWhIf4odBryMq0ML/RnbA5vTg8fkIshiIDrL8rQhVi1FPsNXYYEpX8+jaqdWFFU5+WLev3n1/WLuPewY3V0KOQqGohbojNCLyKxxc2imOtPxKbv1i9cHXP1uczkXtY+iaGMJ5baL5fUtunWOv65XI6gNhvxe3j+WJ7zce3FbucFNud9EmNogt++um+Tw8tCXvzqvf/HDhrkL+PawNCWFWMotqD+CMeo37hjTyB09QLFz/I8x8WAaQPp+U9z7vP5Ay8J+dOyBKfhJ61LwW0fyfnVNx2rN4VwHBViMJYX4n7k0qcmH/Bmgv/gYRfjru62rm5eUO7plj461zrVgMNQPTzDIvD82z0zFSR7/4syMap5pmoTpW5Rx51b1eLMESNZexBDpeefwbplCcSnR6ESiPVqQMSYKxs+CnuyB3k7wW0QIufRdCm/7z9lhD5Ceq1V/t2SBtYoPolBDCusySWq8HWQzcMTAFu9NDmd3FpFt6siOnnMJKJ4EWAyO7NOHb1XWLNtx6TgqvzN6Ohka7+CCu7pHIF0v31tnv/DbRuDxe8iscpEadWF+040mxMZrQxHMw/vowuoBwiaqa+yyFXe7h/bRYzmsThN3lwWrUc0nHOH5al13nHKO7JVBY6aS4ysl9X6/jmp6JzLy3P1VOD0VVTpanFXHH5NXYXbWrRa1ML6Zf80hAom4a4uf1+7i5X1Pe+aPuGLV3s3C27y/ntnOaHUxriwux8sB5LRjTOxmfz0eQ1Yj/IdHiJoOOuJC/JzqKIbKXIKsBo15PVKCZ289J4X+/ba+zb0yQpc53ocTWcHSR1ycR7uBf/w4KheKspBHPsM8eiiqdbN5XitmgIyrIwsuz6j4UZm7MoX9qBKO7NSHM38SPa/fhcHvxN+m5umciLaMDaRrhj9mox2LU1wlRfenXbbx1VWemrcli1qYc3F4fMUEW7j+vOYlhfszfUVDnPatZsruQ/43swNfLM5m1KQenx0v35FBuO6cZO3PLaR0ThK4xr/CHJsEVn0JVgXgBmIMgMFZWLRWKY2TRrgLaxAahO5G+KrvnyUp4dNuDL7WN0HN/NzNvr3Zw6feVPNLDTNsIPWtyPTyz2I5JD+M6m88Kb5xDaRWmY3a6m7wqL1F+x9ino9rA3iUnpmEKRWNCb4C4TrLwYS+WRQ9rqCxWnCZEB1kYf11Xpm/IZtKyvVQ5PZzfOppbz0khMcwPnU5j8rK9hPqbaRkTwKSlGbz061b+e1k7kiP8+WzRHgornTQJtXJzv6YUVzn5c7v4swxuFck1PRLxN+n5YuleKp0ezAYdl3eO55KOcXy3KotRJzqd9jgT4mdk6hYzQy/+GIO9mJKKKvbGGpiw3kHvlHB0Oo2bJ6xi2rg+XNguhvAAE1NWZFLp9BBkMXBdryQu6xjH1R8vA6S094Ql6VzQLoacUjv//nETFQ2kHlUvdNickorcOyWcpWmFdfb7bXMuj1/YigCzgffm7aLM7sak13Fxh1iGdYhFp0FSeO1FE6NeR0zw8fObKahwsGZvMR8uSKPU5mJwqyiu65lIk1A/RndPoKjKyYTF6bgPhCy1ignk/Wu71Cll7lePIXSt7Y3RnkChUJxQ1F3hNKfM5uLjBbv5YH4av93fn/f/3N3gvtPW7KN7cig2p4dJN/XA32IgvaCSn9Zl88nCPWgaXN8zkeQIP6KDzLUqDJTZ3dwxeTWXdYrnh3F90DSNiAATAWYD+eUO/E11xZ9qAswG/vX9JrokhvD6lR3RaRpb95fxr+83YtBr9GkW0fhN2ixB8qNQ/ANKbS427Svlpn7HYYW6QXzi4RTdtk51mS7Rep7rZ+GzDU5u/a0mgq5TlI47OpkJMp9dIg5Ay3ARb1bleLgo5RiFnOh2sGOWmICeRhNWheKUERApP6cpMcEWbu7XlMs6xeHxQqi/EbOhJgpxaNsYcsvsvDBjO6lRAVQ4PDw6bQPT7+lHm7ggbE4PhRVOvlmZwebsmgjmL5dn0KtZOJd1imdQq2iKKp14fT4W7MjHB6xIL+Kuwamn4Ir/PnanF52msd9h4T8zihnZtQnBViND27r4dlUmaw9ENmUWVZFVZOPCtrFc2S0Bl8eHXqexPrOEkeOX1vKJ6Z0STqifkU8WpnFF1yZMXJJe533NBh09mkrKnMmg44e1+3hxRHs27iutI/zc0q8pYf4mbu7XlGEdYg8KORajGFUHWY3kVzjYU1CJxaAnzN90XNPbiiqd/N+MrbV8bXblVfDNigy+H9eX1KgAHjyvBdf3SqKkyoXVqCcswEREQN1qpBH+ZtrFB7FpX93I+HbxQYQ3orQ8hUJxclBCzmlOfoWDD+aLP0tGUVWDubYAJTYXTSMCSI0KZGtOOW3jgnB7fCzdLasYPh9MXp5BXIiVuwel8tRPm2sd73B7Wb6nkJ4pYYT7m2gXHwzIg/T63kmMn1/XJybAbKBJqJU9BZXsKaisVS6yervnSE50CsVZxLK0Qrw+aB8XfOLeJH+HVIlpcUG9m5OCdDzXz8K+ci+5VT6i/TTiA8/e6LIwi45oP43l2W4uSjlGP6+YDvL/9IX1l+lVKBSnHZqmNViBKNhq4N0/svhjex7X907i00V7cHl8ZJfY+HRROot21R+dXFLlorDCQZXDw47cCmxuD3pN47JOcRj1Ot64qlOjm4i7vF5aRAfi8/lYvbf4YHr+4ewvtfHhwrSDBTcmju2OQa/j00V7aok47eODuXtwKmU2F9tzyhnRpQk7c8tZsrsm0sZq1PPZjd2ICRKhIzLAzCMXtGRtRjGf3tCNGRv3syajmDA/E5d3jqdbchjBVvlcAy3iRzNxSTq55Q5uH5DC8jVZvPPHroMeOl2SQnh9VCeSI45PitK+4qpaIk41ZXY3/5u1jddGdyTQYiQp3EBSeD0nOISwABPvX9OFGz9fedBwGyAlwp/3r+lCeD3ij0KhOLtRQs5pzsr0GoO3r5dnMKhV1MFVkMMZ2DISr9fLK7N38tgFLbn7q7XEhVh58Yr2mA16yu0uQv1MzNmaS1SQhQfPa8Eni9IOVqHq0yycOwY046Fv1zP19ppKECaDnrF9m5KWX8nsQ/x3Qv2MTBjbg7IjiEv9m0cQ1BjKjCsUJ4FFOwuICbKc2Ai1tHkSPRbW7Ii7xQfqiP8HhWTOJNpE6Fi878iVV+rFLwyCE2GPEnIUijOB4ioXv2/JweeDCUvSeXlkB576cRMfzk/jwvaxDQo5/ZtHEB1oZnNFOSsOjNsuah9DRpGNt+fs4MUrOhDubzrmEtanEpfHx/j5u3j0glaE+ZsoaqAMe1yIH4UVDiIDzDx0fgt8wH3frOXec5sTHWSh0uEmKdyPdZklzNueR5MQK00j/HlgyjruO7c5Y/smk1vmIDbYQsuYQKIDLRgNOnLL7KzPLGH6+v3odBAf6kff1AiCrUbsLg/NowMP+tmU211MXZXJ8zO2AtA1KZQVaUV1rAjW7C3hmo+XMW1cnzqpTX+H2fV4UlYzZ2suZTY3gccwBk4M9+eb23qRXWIjq8RGkxArcSHWxh/VrlAoTghKyDnNOdRHIzbEwgVtY5iwJL3OAzXQbGBU1yZ8umgPpVUu/EwGcsrs5JTZWfNVMTFBZl66ogM3TVh5sLLAOc0jeGF4e0wGHfEhVqZvyObur9cwrH0cEYetHEUHWXj5ig48PNRBWn4lof5GEkL9iAmykFtur1WysxqzQccD57WoZSSnUJzNLNyZf2KrVXndsGc+xHRUHk7HQPsIPfMynORUeonxP8bPLaa9iGcKhaLRo9M0TAa5B/y5PZ8qp4fXR3ei0uGmdWxQnbR0kCiSW/o35emfN7NwZ010yR/b8uiWFMqDQ1uSXlh54isVHmd0msbqjBImLdvLYxe04rFpG+rsM6xDLImhVt6/tguVTg9fL9/L/43owPvXdqHC4SbAZGB7Thn/+mEjJoOOd6/uwsr0IkptLhxuL//7bTtmg44vbu5Bz6Y1ISu5ZXbunLyaNRklB1+buzWP/s0j+PfFrQm0GIkOsuD0eMgvcVLpcB8UcUBMll+dXddPEiC71M7O3IrjIuQcyetO0zT4G9nKUQcWezolhv6DlikUirMBNdI/zemeFIqmSYrSZR3j2Z5bxptXdmJo22j0Og2dBoNaRvHuNZ3ZmFVK96RQLCZdnRSsnDIHk5ft5e2rO9PiQMnDhbsK+HZ1JoEWA+O+XMOPa7O5d3BzbuqXTKC17gpCqL+JFtGBXNAuhp5Nw4kLsaLTacQGW5l8S09u7JOE1ahH02Bgi0h+uqsvTcOVw75CAZBVXEV6YRXtm5zAtKrstWAvg9hOJ+49zkDaRujRgEVZf6N6VVwXKE6XEs0KhaJRE+5v4rpeNZW6VuyRqkrP/ryZx6atZ9JNPbm0YyxGvYamQb/UCL64qQdZxbZaIk41q/YWU1LlpNTmIsSvcaVWhQcYGdW1CZOXZZBfbue9azrT9EBKUqifkXsGpTJuUCr/N3Mrd0xew0NT17NqbwnFlU7C/E1MX5/NdZ+u4O25uxjQIpJ3r+7Cp4vSGNQykpwy+8H3Meg04g+rFDVvW14tEaeahTsL2FtYRVyIlYIKB8//spX7v1nLjI37a+0XaDEcTPWqjw1Zdc/9dxjaLoZAs4GAehYsL2gbQ0g9Y2mFQqE4XqhQidOciEAz397eG6fbi16n8dXyTDZklXB553jeuqoTGhor04u45+u1JIX78+qojlQ5PUTXE4Y5Z2seu/IquKFPMr1TwnF6vCzcWcDOvAoeu6AVJVVOvludxdb95TxzSRuCjuEBFBdi5V8XteH2Ac3w+UR4OpbjFYoznQU7CtBp0O5E+uOkzYeAaAiKPXHvcQYSZNZoFqLjzww3I1se42Qrtr2Uat79B4QfOZ1NoVCc3uh0GsM6xPHD2n21zIzLHW4Sw/3Qa3D/kBbcM7g5DrcXl8fL/hI7ny7e0+A5Z2zYz8NDW6JvZNU73R44r000c7fm8ersHXRNCuG5S9sQE2zF6fayp6CSOyatJqOo6uAx4f4mskpsvDhzK89c2pZBraLRgPTCSlweL5d1jGf2llyJ1jYZeG32dl4c0YGYQ8asRZUOJi2rW8a9mglL0mkVE8iyPUV8tyaLdvHB2A4rxuFwewmyGCiz1y/Ox4dY8fl8/6hKo83pxs+o56Ur2mNzeQjxM/HNigzmbM0jzN/Ew0Nbqoh0hUJxQmn0d5j33nuPV155hZycHDp27Mg777xDjx496t138+bNPP3006xevZq9e/fyxhtvcP/995/cBh8jVU4Pr/++gyW7C7lzQApOj5dKp4fJyzOYvDyj1r5Ot5ecUhufjOmGzelhUKtI5m3Lr7VPemEVJVUuzAaNUeOX1zKiq2ZtZgl3DWp2zEKMyaA7LqGqCsWZyIIdeTSPDjxxAzu3DTKWQtNz4CwrIX486Byt59c0F06PD5P+GD4/ox9EtYWds6HHrSeugQqF4qQQE2zh/y6XKknztudh0uu4rFM8eeV2NmWXUe5w0yI6gAenrier2Ma4gc1webwNns/p8TbKsZHL4+WPrbn8b2QHtuaU8cfWfL5dlcXtA5rx1E+bWFtPxMwt/ZsydWUm67JKKalyEe5vIru0ih5Nw7l5wspaY87YYAtf3NSDpAg/DPqaBAGfT8azDeF0e5m4dC97Cyt5++rOPDFtAzf1bcqn1IhpP6/LZnT3BD5ZWFdgC7IYCPaTalZRf9OzqNTm5Kd12fx3+hZcHvErMBt0PHpBS0Z1bUKbuOCDJdQVCoXiRNGoU6umTJnCgw8+yDPPPMOaNWvo2LEjQ4cOJS8vr979q6qqSElJ4aWXXiImJuYkt/bY8Xh9fLsq86Cj/6JdhQxtG93g/pd2imP8/DRumbiKaWuyeHRoK67rlYj5QL53sNXIkxe1ZkzvJCqd3npFnGoOXWFRKBT/DJfHy6JdhXSIP4HROJkrwG2H2I4n7j3OYLpG66lwwbLsv2F6HN9NvImc6r6pUJwJmAw6PlqQRpifiWEdYvl88R6e/mkz901Zx4cLdrMzt5w3r+zE4FZRLEsrZHCrqAbPdUmHOIyNcLQdYDbQq1kEL/66jQmL9xIRaKJphD9fr9jLw+e35Iou8RgPiN4RASYev6AVNpeH5XvE7HldZgmdEkPo0yySu75cU2fMub/UzqPTNlBpr33PDfEzcVmnuAbbdW5r+cznbM1j2uosLu/ShOxSG31Tazx2/tyRR5vYIC5uXzs6NTLQzGujO/H23J14vH+/ourO3Aqe/mnzQREHJArov9O3EhtiVSKOQqE4KTTqiJzXX3+dW2+9lbFjxwIwfvx4ZsyYwWeffcbjjz9eZ//u3bvTvXt3gHq3n24UlDv4fHH6wd837ivlrkGpNI3wZ89hxsLxIVbaxwfzym9i7jZnax7X9EgkxGrk9dGdiA4yExtsISbIgl6vo6Sq4UpTIPnFCoXi+LAqvZgKh5tOCSEn7k12z4PQZKmkpDhmEoM0ov00fk1zcU7CMd7/EnrA6s9EzGl54YlpoEKhOGlEB1loFRPIt6uzCLIaawU5ZhbZ+H1rHrvzKxnVtQl+Zj3RgRa+X7OvziJYQpiVc1pE8vvWPK7snoCfqfGMrXQ6jVYxgYT4GVm9t5jd+RUHt323eh+TburB+W1i8Ph8VDndTF2VxYo9NZVWo4PM+JkMbMspJ7+ifr+atRklFFY6CT2kwIZep3F553i+Wp5Bdqm91v4pEf4kR/gfTHv7fWsuH4/pxrjJa/jfyA70SA7juzVZlFS6mLUph3sGp3Jdr0Qyi20EmA3YnB5e+nUb8aFW/P/m36LC4eL9Pxv2RPt44R5eGdkBi1H/t86vUCgUR0vjeaIchtPpZPXq1TzxxBMHX9PpdAwZMoSlS5cet/dxOBw4HDUPoLKysiPsfXzx+HwUV9WuTvXkDxt5eWQHVu4p4tdNOXh9Pi5qF0uf1HAe+W79wf1aRgcSG2Ll15nb6NAkhNSogFpGe2H+JjonhtQbGhvmb2qUYcCKM5tT2Rf/KX9uzyPEz0hyxAky/7aXQvYaaHXxiTn/WYCmafSK0zNzj4vn+lmOLb0quIn8bJt+Vgg5jbkvKhRHQ5i/iecvb0fXpFB+Xp/NmN7JLEurESmW7Crkul5J7Mqt4KsVGdicHl4e2YGluwuZtXk/Ph8MbRvDkNbRWAw6fly7j8s6xR13IedE98WYYCs3923K3K21I90dbi8TlqYTGWBm4tK6fjZ6nUbf1AgAym1HXjh0uutGQcaH+vHtHb35ZmUmP6zdh07TuLBdDN2bhvHodzXVs6rTsJweL/dPWUeXxBAeHNKCbslh6HUa13+6nN35lQSaDdjdHlweH3qdxltXdfrbPo4Ol/eIUet7CyuxuzxKyFEoFCecRhjsKRQUFODxeIiOrp1qFB0dTU5OznF7nxdffJHg4OCDPwkJCcft3H+Fn0lP9+Taq+uFlU5umbiKTdmlvDqyA5/d2J3+zSN49ufN5JY5DuRyx/HO1Z0J9TMy5bZenN82pk61hFB/E6+P7kh0kLnOe352Q7daxnMKxenAqeyL/5Tft+bSsUnIEUuV/iPSFwI+iOlwYs5/ltA73kCpA+Zn/o3qVQm9YNtM8PyNYxsZjbkvKhRHS1SghVv7p/DZDd3p0yycq3vUfM+dHi+/b8klNcqf7smh5Fc4uHniSrbllDGmdzI39ElmZ245q/cWkVlcRauYE+OPdjL6YqvYIG7tn1LndX+Tnlv7p9SJNDXoNMZf1/Vg0Y3EI6QZ+Zv0BDcgqMSH+nHfuc2ZMLYH1/VKYuO+Um6ZuIqiypoFTk3jYLl4gHK7m27JYSSE+REXYuWzG7sztG0MlU43Lo+PNrFBTLmtF6lRAcfyEdRus9lAxyNUn+ycEIKfSYk4CoXixKP5fL6/nyR6CsnOziY+Pp4lS5bQu3fvg68/+uijzJ8/n+XLlx/x+OTkZO6///6/NDuub7UjISGB0tJSgoKC/tE1HA1rM4q54oMlHJ7KmxTux4PnteDJHzbx0oj2JIT54fR4MRt0hPmbiAwwYz6K1YDsEhtb9pexPrOEZpEBdE0KJTbYUst4TqE4HTjVffHvkpZfweDX5vPgeS3qCLPHjRkPAHroOubEnP8s4skFNpqF6Pn4gmP0OMjfDjMfght+EcPpM5jG2hcVin9CcZWTnFI7c7fmomka57aKItzfRLHNxWXvLsbmqh1ZEupn5MtbevLQ1PW8fXVnmkcHHvc2nay+WFLlJLfMwdytuXi8Ps5tHUVssJVQfxP55XYyimys2FNIRICZnk3DiAqyHIxIKbW5+M8vm5m2Zl+d8/774tbc0CcJo77h8WpRpYObJ66qN4J8SOsoHrugJTtzK2kSaiUmxFLHwLjC4aK40oXH6yPQYiA8wFznPMfK9pxyLnp7YR2fHZNex6/39afZPxCKFAqF4mhptKlVERER6PV6cnNza72em5t7XI2MzWYzZvM/v+n/XVrFBvHNrb14bvoWNmeXYdRrXNw+liu6NuGBKeuocLh5/PuNvHxFewa1ijrmsN24ECtxIVaGtG7YRFmhOB041X3x7/L7llxMeh3tT5TRcWkW5O+ATtecmPOfZQxMNDBxk4ucSi8x/scgaEe0kNLvm38444WcxtoXFYp/QqifiVA/E61jawskQVYDU2/vzfMztrB8TxE6Dc5pEcm9g5uzZX8Zz1za9ohRKf+Ek9UXQ/xMhPiZaBlTV4yKDLQQGWiha1JovccGW408fmFrmkb48/HCPZTaXMQEWXjo/Bac2zr6iCIOQJi/mXeu7sx936xj9d7ig68PaBHJf4e3IzbYSvPohkWrALORAPPfS6NqiKRwPybd1INHp20gq9gGSOTRa6M7khCmrAkUCsXJodEKOSaTia5duzJ37lyGDx8OgNfrZe7cudx9992ntnHHEatRT4+UcD4Z040KhxtN0wg063F7fXx6Q3d8QGSAicggM6a/eBgqFIqTz6+bcujQJPjE5cvvngtGK0S2OjHnP8voG2/g660uJm128kiPY0gx1TRI7gebf4QL/wf64ztxUCgUpydmo4F28UG8ProTlQ43PnxYjXp8+IgPjTyYYnQ2Exlo5s4BzbiiS5MD0eN6ooPMaEeZbtwk1I+Px3SlsMJJud1NsNVIeICpjm3AycJi1NMnNYJpd/ahpMoJaIT6GYlSf2uFQnESabRCDsCDDz7IDTfcQLdu3ejRowdvvvkmlZWVB6tYjRkzhvj4eF588UVADJK3bNly8N/79u1j3bp1BAQEkJqaesqu42iIDamr8MeHqvKGCsXpTHaJjXWZJYwb2OzEvIHPA7vmijeOEg6OC35GjYEJBiZvcTKusxl/4zH4GjUdAJumSQWxFuefuEYqFIrTCk3TiA9VkRhHQq/X1TuWPVrC/M2E+Z9ekYDRQRYl1CkUilNGozZCufLKK3n11Vd5+umn6dSpE+vWrWPWrFkHDZAzMjLYv3//wf2zs7Pp3LkznTt3Zv/+/bz66qt07tyZW2655VRdgkKhOIOZuXE/Rr3WYMj5P2bfGqgqhCbdT8z5z1IuSjFQ6YRJm51/vfOhhDaVEvDrvzoh7VIoFAqFQqFQKKCRR+QA3H3pM5eoAAEAAElEQVT33Q2mUv3555+1fk9OTqaRejsrFIpGyE/rsumUEHLcS84eZPuvEBQnP4rjRoSfjgGJBsavc3J1axPB5qOMytE0aDYY1k6GqiLwO0Hm1gqFQqFQKBSKs5pGHZGjUCgUpyu78irYuK+UfqmRJ+YNKnIhayUk9BQBQXFcGdHCgM3t4701jr/e+VBSBoHPCxumnJiGKRQKhUKhUCjOepSQcxphc7nJKbWRW2bH7fGe6uYoFIp/wLerMwkwG+iUEHJi3mDbdDCYIbbjiTn/WU6YRcelqUY+3ehke5Hnrw+oxhoKib1g5aegIkAVCoVCoVAoFCcAJeScBni9PvYUVPLUj5s5/40FDHt7EW/P3cn+EtupbppCofgbON1evluVRb/UCEyGE3CbdVbCjlnijWM4vcwfzyQuaWYgxl/jkT9tuDzHIMq0vBgKd0pFMYVCoVAoFAqF4jijhJzTgMziKi57dxHfrc6izO4mv8LB23/s4obPV5BTaj/VzVMoFMfIrM05FFY6Gdwq6sS8wfYZ4HZAYp8Tc34FAEa9xh2dTGwq8PLm6mNIsYpuB+GpsOTdE9c4hUKhUCgUCsVZixJyTjF2l4cP56dRZnfX2bYjt4K1GcWnoFUKheKf8OmiNNrFBZEQ5nf8T+6shE3fQ3w3sAYf//MrapEaqmdUSyPvrXXye7rr6A7SNGg7AtLmQdbqE9tAhUKhUCgUCsVZhxJyTjElVS5mb8lpcPu0NVk43covR6FoLCxPK2R9ZikXtIs9MW+w8Ttw2yFl4Ik5v6IOl6Ya6B6j5965NjbmH6VfTlJfCE6AeS+c2MYpFAqFQqFQKM46lJBzitHpwGLUN7g90GJAr/5KCkWj4c05O0kO96NLYsjxP3lpFmz5AZL7g/UEnF9RLzpNY1xnE/EBOsbMrGLH0Zgf6/TQ6Vrxydk978Q3UqFQKBQKhUJx1qAkglNMhL+Z63slNbj92p5J6HXqz6RQNAbm78hnaVohI7o0QTveJcG9blj0BliCVTTOKcBi0Hi0p5lgE1z5S9XRReYk9YWoNjDzEfE0UigUCoVCoVAojgNKITjF6HQal3WKp2OTul4X1/ZMJCXC/xS0SqFQHCtOt5f/Tt9C69hAuiWFHuez+2DlJ1C4A9qNBL3xOJ9fcTQEmjT+1dtChFVj9M+VzP4rzxxNg17joDgN/nzx5DRSoVAoFAqFQnHGYzjVDVBATLCFj8Z0Y0t2Gd+tzsTfbODqHokkhfsT5m861c1TKBRHwbvzdrGnoJIXhrc7ztE4Plj7JWz9BdoMh9CGI/gUJ55Ak8a/epn5YK2T236zcUcnDw91M2PUN/A3D02GTtfDojchsTe0GHoym6tQKBQKhUKhOANRQs5pQnSQheggCwNaRKJpHP+0DIVCccJYsruAd//YyeWdm5AUfhyj6BxlsOwD2LMAWlwAiT2P37kVfxuLQeO+biam73bz8XonCzLd/G+glXYRDfidtRsB+dvgu7Fw/U+Q0P3kNlihUCgUCoVCcUahUqtOM3Q6TYk4CkUjYkduOXdMXk3buGBGdI4/PietzId1k2HaLZC1EjpdDSkDjs+5FccFnaZxaaqR//SzUOHycen3lTy10EaBrZ4qg5oO+j8k0TmTLoPtv5709ioUCoVCoVAozhxURI5CoVD8TVamF3HrF6sI8zNx37nN0en+rgjrg+J0EW0ylkv0ht4MTbpCyiAwBxzPZiuOIykhOl7ob+G3PW6+3+niux0ubmhn4qb2JqL8DlkrMVrh3Odg0Wvw9VXQZQwM+jcERp+6xisUCoVCoVAoGiWaz+fznepGNCbKysoIDg6mtLSUoKCgU90cheKs5VT2xZIqJx/8uZtPFu6hZUwgD5zXggDzMeriXg/kbYaMpZCxDCryRLyJSJVKR1FtwGg5MRegOCGUO31M3+3i93Q3bi9clGLgylYmesXp0VVHWvp8EpGz9gvwOKHtCGh7OST3a7SCnXouKhSnB6ovKhQKxdmDishRKBSKv8Dr9ZFVbGNtZjHzt+czc9N+fD64omsTLu0Yh/6oInF8UJEPuZshe41E3zjKwRoMka2h5cUQlgJ6dVturASaNK5ubeLSVCPzMtz8sdfNT7uqiLRqDEky0LeJgS7RemJbXojW9BzY8SvsngsbvgFND1GtILo9hDeDkCQIigX/KPALk7LzBvOpvkSFQqFQKBQKxWmAisg5RkpLSwkJCSEzM1OtdigUJ4DAwMCj8ok6lr44Y3MeMzbl4fOBz+fDB7i9PjxeH26vD5fHh8Plwe72YnN5qXC4cXkavjXGBpkZ0DyMwANROJrbhj5zKZqjFLxe8HnA40bzONDcVRJ9cxg+ox++wDi8lhApU6044/ABuyv9WFkaSKm7/pLxoUYXwTo7/t4KLJ4KjDgx4EWHl366Tdyk/xWDVo/vzuHvpTfjM/mD0R+f0QoGCz69CXRG0OnFp0fT4Q1qgqPPw/j8wv/ynCeiLyoUimNH9UWF4vTgaPuiQnEyUELOMZKVlUVCQsKpboZCccZytCHhR90XdQaSHvnxnzdMoTgFLDDdT6Iu77ie8/bpNj5a7frL/Y57X1QoFH8L1RcVitMDlbaoOJ1QQs4x4vV6yc7OxufzkZiY2ChWPcrKykhISGgUbQXV3hPN6d7eo13tqO6Lh+9/ul/fieZsvn517cf32v9pXzxVnM3fg+OJ+hyPD8fjczzVfVF9F44N9XkdG43p8zpdnnNnAhMmTOD++++npKTkH51H0zR++OEHhg8fflza1ZhQZgzHiE6no0mTJpSVlQEQFBR02t90qmlMbQXV3hNNY2vv4VT3xYZo7Nf3Tzmbr19d+8m99r/qi6eKs/l7cDxRn+Px4WR8jie6L6rvwrGhPq9jQ31ejY8bb7yRkpISfvzxx1PdlLMS3V/volAoFAqFQqFQKBQKhUKhOB1QQo5CoVAoFAqFQqFQKBSK48Lrr79O+/bt8ff3JyEhgXHjxlFRUVFnvx9//JHmzZtjsVgYOnQomZmZtbb/9NNPdOnSBYvFQkpKCs899xxut/tkXcZpjRJy/iZms5lnnnkGs/n0LwfbmNoKqr0nmsbW3mPlTL++v+Jsvn517WfntR+O+iyOD+pzPD6cCZ/jmXANJxP1eR0b6vM6M9HpdLz99tts3ryZiRMn8scff/Doo4/W2qeqqooXXniBL774gsWLF1NSUsJVV111cPvChQsZM2YM9913H1u2bOHDDz9kwoQJvPDCCyf7ck5LlNmxQqFQKBQKhUKhUCgUiqPmWDxyvvvuO+644w4KCgoAMTseO3Ysy5Yto2fPngBs27aN1q1bs3z5cnr06MGQIUM499xzeeKJJw6eZ/LkyTz66KNkZ2cDyuxYoVAoFAqFQqFQKBQKheIfM2fOHF588UW2bdtGWVkZbrcbu91OVVUVfn5+ABgMBrp3737wmFatWhESEsLWrVvp0aMH69evZ/HixbUicDweT53znK0oIUehUCgUCoVCoVAoFArFPyY9PZ1hw4Zx55138sILLxAWFsaiRYu4+eabcTqdRy3AVFRU8NxzzzFixIg62ywWy/FudqNDCTkKhUKhUCgUCoVCoVAo/jGrV6/G6/Xy2muvodOJJe/UqVPr7Od2u1m1ahU9evQAYPv27ZSUlNC6dWsAunTpwvbt20lNTT15jW9EKCFHoVAoFAqFQqFQKBQKxTFRWlrKunXrar0WERGBy+XinXfe4ZJLLmHx4sWMHz++zrFGo5F77rmHt99+G4PBwN13302vXr0OCjtPP/00w4YNIzExkZEjR6LT6Vi/fj2bNm3i+eefPxmXd1qjqlYpFAqFQqFQKBQKhUKhOCb+/PNPOnfuXOtn0qRJvP7667z88su0a9eOL7/8khdffLHOsX5+fjz22GNcc8019O3bl4CAAKZMmXJw+9ChQ5k+fTqzZ8+me/fu9OrVizfeeIOkpKSTeYmnLapqlUKhUCgUCoVCoVAoFApFI0FF5CgUCoVCoVAoFAqFQqFQNBKUkHOM+Hw+ysrKUIFMCsWpRfVFheL0QPVFheL0QPVFhUKhOHtQQs4xUl5eTnBwMOXl5ae6KQrFWY3qiwrF6YHqiwrF6YHqiwqFQnH2oIQchUKhUCgUCoVCoVAoFIpGghJyFAqFQqFQKBQKhUKhUCgaCWetkPPee++RnJyMxWKhZ8+erFix4lQ3SaFQKBQKhUKhUCgUCoXiiJyVQs6UKVN48MEHeeaZZ1izZg0dO3Zk6NCh5OXlneqmKRQKhUKhUCgUCoVCoVA0yFkp5Lz++uvceuutjB07ljZt2jB+/Hj8/Pz47LPPTnXTzipsTg8lVU48Hi9lNhdZRVVkFVdRbncd3Ql8PqjIg/IccNtPbGMVCsU/xuGSPu90ew6+5nR7yC6xkVFUSX6549hPaiuWe4C97Di2VKFQKBSKswOv10epzUWlw113Y1WRPGMdFSe/YceIz+cjp9RORlEl+0tteLyqepvizMZwqhtwsnE6naxevZonnnji4Gs6nY4hQ4awdOnSU9iyo8Pr9aHTaeSV2ympdOEDQvyMRAdZjnic3eXB4/Xhb5Y/uc/nQ9O0Oudt6FgAi1FfZ5vH68Pp8WDW6/EBRZUOfECA2YCfqebrlVNqY+v+MlL8ncT4g0Gnw+Vwk2kz49NbeHveLpKD4IpWfgQFGXGYLXh0RmyGEFxeH1ajDrO7AoO9EJ+9FL0lEJ/Xg95RDlWF+CzBEBwvDxuDCY81EjcaRo8dnckfTaeTbToDWEPBYAH/cDweL26nDZOjCE1nBI8TdHrZx+QnjXdUgqtKjvULlYeZ2w7mQDCYj/6Ppzirqe5jJVVOiiqdONxegq1GogPNaJpGlcuNSa/HZKhfXz+8z/7V60eDzeVGp2mYDXX7djVuewWuyhI8Pg2T2YTBWQ6aBpYQdH6h4PWC20a5S8Pm1kDT0Os0qhxuAvVOzK4S9Hix6wJYV6ChaZASGXDgXqExY8N+KquqiPWHco+JHs2iaBMbiN3tRadBkNWIw+nFDZh1HgKdhRjshejMAWAORMvZCFkr8NlK8XYZgxbUBJ3mw+d24LOVgMGCz+iP5nOheV1olQWg6cAaAkHx4HWDvRTQQG8CtwOMZvALr+nrJn9clUXoAH1AhPR7r1c+IN1ZuR6iUCgUZxRejwfcNnA58JisUFUCXg82UzhGrw2Tz47O7QB8Mh7UGaXMu96AzlkOPi9eoz8uSzQeZyV6jw2d2yaLDdYQHJZIsJeiw4vOEozNqyfAXYzBXQkFO2S/2I5UWaJJq7JiNekJMOsJ9FWitxWic5bhMQVRoQ+hXPNHp+nYX2qjsNJJamQAUUFmwvzNeLw+9pfYWJ1RTESAGafbw5yteQRaDFzaKZ74ECvBViNOtxev10dhpYOZm3KYuWE/fmY9N/drSouoQMK0cqy5q9EWvgJl2RDXBQY+QVVQU3JtOhwuD4EWA1FBFoz6o3sO7iuporDCyW+bc6h0eDi/TTTNogL+cg5zNBRVOvltUw5vzNlBXrmDcH8Tdw5sxvDO8UQEqLG64sxE8/l8Z5VcmZ2dTXx8PEuWLKF3794HX3/00UeZP38+y5cvr7W/w+HA4ahZJS4rKyMhIYHS0lKCgoJOSptLq5zkljuYsyUXo16jR9MwCiucaBrgAzSNJiFmgv3MWIw6bE4PRZUu3F4v/mYDRr1GfrkDp8dHdKAZk0FHqc2F1aTD59OYuy2PNXuLaRblz8iuCeDz4fL48DPp8PnA7fXh88nczajTUenyYNBp+Jn0ZJfYcLp9RAaa0Wk+csoceH3QKlxHoKcMPW4ozUIXEA1eJ5q9DNIXQmAsxHbCl7cVX0IvNJ8bFvwPLW0eWEKg+y0Q1xmnV8cGVxPahDixrnwPzeQv2xN6iuCSvhgSe4HLBrYiCIwBZxUERIE5SAQbWwHojDJJqyqC/K34CndB6nkQkggeF5rbAUYr+DzgdePzj5KIH69btrsqQW/EZw6Egl1o+MDthKA4qMyTCV9EcyjNgn2rITQRmvQAvRl8bjm3X/hJ+b6cqZwOffFIuD1e9pfaWbK7gB255QxpFU1ShB+VDg/7Smz4mw0EW408/eMm9DqN63snER/ih14Hep3Gm3N2otdpjO6WgEmvERtswePz4fGCTtMwGTR0mobN5cGo11FY4eS71VmU2V0MaR1NYpgfIX4GIvxNaJqOcoebgjI7TcL9sDk8OL0+DDoNDcgts2M16THrNYwGPaHeIqzeCvQ6A26DHyVaIJqmJ8yRhbbgf2iZy+DiNyBzOaz6FJwV+Ib8B5oPgaK9aHv+xJcyECKa49OZKfH5E6izoy/aia4sC7b/hs8/AmfHa8nQ4vlwURa39U0k3lyJVQ+a14nmqsKHAazB4CiFgp0itgTGHoi408AchGYvBWclGC2wYxZUFEBCN4hoCQFR+Mr3o/l8cp+xhkFyP9izAAp3yb2j1YVgCgRNh8/rBp9XRFyzv4i9Xg+Y/PEBeN1oHgc+k7+c016Kzxwk94ayLLn/hiaimQIBH5RkiMjjHyX3IP0BMdtWChU5sHseeF3QbLBcl1/YKfmu/lNO976oUJwtnG190evxyBjPbUPzuuHA2Ay9FTQ9OIrleaG3yP3XeSCKRG+G4nQZ0wXHy++2ItjxGxj9oO1wtLJsWD0BHOXQ5lIR+pe8I8+2NsPRds+F5R9ASSZEt4NBTx5YDHDBig9h43dwydv4dAa0dZPB44KWF0BQAuh0+PK2oq2ZCPYyPL3GQcpA9LZimHaTjGEP4Gt+HiWDXuaiz/fw040pRP75KNrO3w9u9zQdSNWFbzH0091kl9ZEow9oHsnLIzuQU2bnmo+X8eqojnyyMI01GSW1PsO7BjXj4vaxfDB/N2N6JXP312vILasdDfvqpU25tHwKpqVv1v4DaDpKR05l9O8mtudUEGg2cN+Q5lzRJZ5Q/yOLJRlFVUxdmcm783bVer1HchhvX92JmGDrEY8/EnaXh48WpPH67zvqbLuxTzIPn9+CAIvxb59f8fdIT0+nadOmrF27lk6dOh3VMRMmTOD++++npKTklLajsaCEnAM0JOQ8++yzPPfcc3XOc7IekmU2Jwt3FnDvN+vQ62Dq7X0oqXICsG1/OWjQPzWC3DI7ny9J59b+KazeW8zni/dQZncTEWBi3MBUuiSFYNRp2Fxe/jN9C16fj5dGtOePbXmkRARQ6fSQEGplxsb9/LB2H1VOD80iA3jsgpZkFFWxNK2QC9rG0KFJMK/9tp2sEjtjeicB8OSPG7lncHN6p4STmVfIedGVBK18A604HYa+CMveh+0zwRQAl7wlD9LN38uEp/P1MtnSG2HvEpl8Za8VYeTKSRAQjU9vAmcFmilAXneWy7nMQSKa/HKPTOxAVtq73wKhKbBiPPS8DZL6ywQ0NEnElN3z5IHfbCDkb4fwZuBxi4gTEAW2Etj4rTyE214OJn+YdotMKnveAT4v/PYkxHWSB7k1XI7dPQfmvwKB0dCku7xHx6ukXUartNlgkUGGTi/tt4bIe5fvlxWZqkIZIARGK+HnME51XzwSXq+PdZklXPvJcsb2Tebi9rGYDDoW7yqg3OGmVUwQm/aVkldm5+b+KVQ53bw+ewfzd+bj80HnhBCevbQNYf4mvD7ABxVONy//uo2Fuwrw+aBPs3AePr8l/iY9X63IYOLSvQzrEMslHeNwe3z4m/WE+5vJK7fRKtIPg0HD6rNjdBSDy4bOaKbCrcNljiDK6kVz28HjgNzNaL89ISKkpkGzIfgG/QsswWgL34CE7hDTHrJXw/z/iUgx9AXYNhN2zwVLMPS6E0KbwtJ3IaYDvlYXo638GNLmS3Tb4CchIBYqC/BENEdnK4Y1E9H2LhHxtdc4ETY0HZRmyEC6YAes/EQG7Ze8CasnSr/qdI186LYSiZBLXwKL3xBRtc+9MkgvSpd22csgZwOEJkNwEyjbJ9eiaZKWaS+V1z0H7knOckgZJCLtmgnQdgTEdoD1U2W1tP0VsOIj2L9e2tv7bkjoIW3UNOnPjnLZNyBG3tdRJsf8+WLtL03nMXDu0xAQefK+qMeJ07kvKhRnE429L3pdNjS3W8ZFznIRRaqjIw1mec1RIffV7TPRPE5oMxz8I2D205C/BSJayDMoJAnmvQhZy+HyD+H3p+V+H9Ua5v5Hoi1BomkGPCrjxsVvyTiucCdsmFq7ceHN4IKXYMdsGbeu/rzuBVz2vuzndkBlvghB6Qtr7zPiI1gzURYeq7n2WxkDThwmY83D8PV9gG0pN9J8ySMYds+us93TdAAT45/lP3P213r9ht5JOD1e0vIr6ZYcxnuHiSbVjL+uK3O25uDzwbQ1+2pt0zT448YEmn5zjixaHE5oMrN7fcFtP2QdfOn54e24pkdig5H95XYX6zJLuP7T+gvLPDq0JXcMaNbg8X9FZlEVQ16fj8PtrbNNr9P446EBJIX7/61zKyAzM5NnnnmGWbNmUVBQQGxsLMOHD+fpp58mPLzhuYrH4yE/P5+IiAgMhqNLArLZbJSXlxMVFXW8mq+EnDMJp9OJn58f3333HcOHDz/4+g033EBJSQk//fRTrf1P9WrH7vwK3p27k4s6xGHQQXyIlR/WZjN+we5a99cxvZPo3zyCmRv388Pa7DrnuXdwqkz63l3MwJZRPDK0BXsLq/jgz92sySjm+eHt+H7NPlbtLa5z7CsjO/DJwj1szy2ne3Iozw9vx8VvL8Lt9TGiczz3DU5Bq8jF6KkiypuH/uvRInYM/wAWvS4r6wBXToYFr8gk6FBaXwrdb5WQVq9bVjjMsmJOSaZM6JoNkhWPNZMkzcloha43iZjy4x2yin4oF70ik8A+98okK7oNbJwmqzcpA+TBuX4qXPyKrKKs/hz63g/522D917XP1WwIDHkGfrwTcjdB+1EQnCDXpung8vGw6A2ZuJ33HxFj9EYRbox+ci3Z62QFP6EXTLwEHCVy3vOelZWkeS+IaBSeUvPg9I+GwMY3yTtRnOq+eCT2l9h4fsZm7ju3BV4fbNlfxqPfbcB9SH720LbRPHheC3JK7Tw4dT2Flc5a5zDpdXx6YzfumLSaljGB3HZOCvO25VFud9MlKRSzQYfd5aFzYigjxy/l3xe3JrfMwRdL03G4vZgNOiaOSqJbaAV6r1OizywhaEGxUJEvK5NBcfiKdqNtnyURZDHt4MtRdS8oOAHG/AylmVBZIP3SP0LEEr0JPj0fqgpqH9NmOPR/SAanJXvBXgJoEN5cIl2C4sFkkQHvdzdDziH3gWbnQp974NdHpb8DxHWGgY/Dny9B3lYY/QX8dBdc/5MIoV6X3Ge8HjCYIG+b3F/yt0nf6/cAJPWRflucLtE4V3wCxgDpu7t/h1GTIHuN/H4ocV2g/4Mw5TrofB1EtgL/SLnXHP7I7H4LdLpWrjFvMywfL5+ZfwT0ewhSz4X3etT/xRk9SVZ+7aVyHZaQI6dquWxyf/H55L5iDWl43xPI6dwXFYqzidO9L3o8HjRHKZrLLuMuj0MEFLcdNk2TqJo+d8OKj2HNFzK+i24nzxJHOUS3lfHZ2sm1T5zcH7pcD9/fVvPahS9LJHTxXvjjP1CRK/f8r6+uv3FXfArz/g/OfQq+vbH+fbrfIgtyn55Xv6jhHwHXfi/P1/L9MO3m2tutoTIenXZLzWtxXWRBwueVZ159WELw3jwH3Xvd6t8O7L16PgM+ry3CmA06XhvdkYIKJx8vSGNfia3eY0d1bcJF7WO5+6s1VDprj5+jg8z8PLiI6Fm3Nvjeu0bNZcik3IO/h/ubmH5vP2IbiKrJLbPz1pydfLUio97tMUEWfrq7799OsdqQVcKl7y5ucPu3t/eme9PGGQF7KKVVTgoqnJTZXQRZjUT4mwj2M53Q90xLS6N37960aNGC559/nqZNm7J582YeeeQRnE4ny5YtIyys7mfrdDoxmU5s246WM1nIOeuS+00mE127dmXu3LkHX/N6vcydO7dWhE41ZrOZoKCgWj8nk5JKJ1FBFu76cg2/b8kjo8jGB/N313mefLF0LzHBVn5cV1fEAfhoYRrFVS4+HtONVjEB5JQ5uPWLVazaW4zVqCfEz1SviAPw7rxdXNsrEYCV6cWsTi/mul4SjeOvcxKXN5/Eb88ntmQN+pkPycPJP0JkfY8bhr0J106TaJRBT0LXG2VbNVt/llQKTQe7/5AVine6yKRJr4em58DCV2H5h/KQB5nQLHtPUiu630IdVn0mr4c1g9zNMOFiWP0ZbJgCP46DtZNklX/BqzKJ0xkgKLauiAMSaZO1QsSmaw6szCf2FLHG55UVoP4Py2T3h9sBH2z9RcJ68zbLSr81SNIprBFwy+9w51IY9C/IXC2rOIP+JdEFeVuhPFvCfTOXQuFuWcXJ3waVhQ1/Uc4CTnVfPBIlVU7uObcFr/y2g/2ldh76dn0tEQfgt825bMkuY+O+0joiDoDT4+Wr5RkM6xjHmowS7vxyDee3jSEl0p+deRVEBJppER3Id6uz6Nk0jECLgV15FcSHyMDp4xEJdHetxLB9Btqk4WhTr0f74hL57ruqJGVpynVoU8fA+q+kv815tnYjNB30vguu+gqKdst3PDQRNnwNX14BHw0U0XLEhxK9dihbfpTB7NK34atRsOhNEVS2/AgF26AiW16b+xx0vBJGTZRoHKMf9B4HX46sEXFAIvO+uxkGPyWD/+J0uOUPETq2/ix9fN1XImwUpskAOn+b3C9GfibXsuVnGPIcjPhYxJ6f7pIUqt53wa3z5P0PF3FAxJ2sldB0gEwgEnrJSmt4al1frFWfiqiUvgj+eF5EHJD///YEvlWfidBTH0vegsyV8NWVMOlyWPqOiNf1UZIBMx+Ve+Ob7URkytko0UMnmdO5LyoUp5L3/9xF8uMzWJZ2cp7Xp1tf9NrL8ZVm4yvYiW/bDHRznkG37ku04t1oZdloFbloqz9H+/wCidpseSFMu1UE8OrxXe4m+G6sCNWV+XVFHJCol9J9ENup5rV5/yep7mHJUJQGrS6WhbqGWDtJngU760a8HGTz9yLgNLTmXVlwYPyqwYZv626P7SjR5ofiFyZj2JL6RQ0A7CUSfXQE/zujq67Bv8PtRUPDYtRR7mi4cEi5w41ep1GfF7Db45NI+CPg02pPHwsrndgOE4QOxev1UnaEQialNhfefxBXcCSfPwA/85G3NwayS2zc/fVazn19Ppe/v4RzX5vPPV+vJbsBse54cdddd2EymZg9ezYDBgwgMTGRCy+8kDlz5rBv3z6efPJJAJKTk/nvf//LmDFjCAoK4rbbbiM9PR1N01i3bt3B8/388880b94ci8XCoEGDmDhxIpqmHUylmjBhAiEhIQf3f/bZZ+nUqROTJk0iOTmZ4OBgrrrqKsrLyw/uM2vWLPr160dISAjh4eEMGzaM3bt3n9DP5XThrBNyAB588EE+/vhjJk6cyNatW7nzzjuprKxk7Nixp7ppdUgrqOTDBWk4PV56pYQxZVX9g3yzQUdemb3BZ43d5cVk0HHfN2vxNxv5zy+b8fogKtDMI0NbsrewssE27C2sIvyQ3Nfv1mTRNzWcQLOBBzprGL+7Xlb//cJlJR4kzaIsFy76Hyx+UyaBX46Eb66WB+LFb9S8QXATSddY9blE0TgOdM6MZRK94vOIAFMfm3+Q1fzDKcmEsFSJClgzoe723C2yIrT5e/m9xQVyroZYPREq9kv72wwXUSWihaysXPB/ssIU2RJ63CaTw07XQdEeMU/e/D18fpGIPJ4qGWzkbpaIBq9T2ugsl0nptJtlUle0SwSj9MXyM/9/8tnaSuq2rTxHUsQKd8sxipOOxaTnzd93UOVys+BAulR9bM8tZ/meogbPs3pvMa1iAgHpJi/M2Ep4gJkpKzO5c/IaduSWkxBs4JNLwhla8SPjQyYxpU8Wc29OoWdAPnoN6W8el4ipF78OF70q0WD2Uuh3PwREy5uFNZXvYTWaBpe+I/3vowHw7Q0ysJ4wTNIRfT7pM2snwcxH4ML/1b2ArT9D/g6JThvwKEy9TrxhKvPhi8tE9Nj6C/z2L/j9KbjsPWg/Wgbq3nqqZTgrYNccSXcqyZTfczZJZJBOLwPiwBgRdAESe0u65pTrRDBa/oFcx7wXYPiBiULxHvj6SokUrE+4rWbDFGhzGaQMFKG3w2jocCWM/FxSovQH8u19PlkB3vZLvafRVn0KrS+p/z0qC2HvIshYCvvXSSrAZ0PrDvBL98m9cO0XNekB6Qvhk3NFcFMoFKcFm/fJ5Dotv+Ex1ZmGt7xAhJu1X6Gb/yJa+gK0oj1ofzyPtvQdmP1vmHyFjKEylsqiGUi0is4ownl97Fvb8NgPDtyjL6353V4qwnb189c/UqJKG6IkQyJIPUcQwz0uuf8fCb1J9vHWcx6PUyLID6Vwt4yXE3s1fM7Ilnh1+oYFJMBlrCvaRQSYsLncrM8soU+ziAaP7Z0SzqKd+ZzXJrrOtsJKJyUBzeS66iO2E6vyagtMFqOuwUINAEa9nl4pDafg9G8eQYD579ffCQ8w0SI6oN5t8SFWIv7Cv+d0p7TKyWPTNrBwZ+1I6AU7C3h82gZKq07Mgk5RURG//fYb48aNw2qt/T2OiYnh2muvZcqUKVQn97z66qt07NiRtWvX8tRTT9U53549exg5ciTDhw9n/fr13H777QeFoCOxe/dufvzxR6ZPn8706dOZP38+L7300sHtlZWVPPjgg6xatYq5c+ei0+m4/PLL8VYXpTiDOeuqVgFceeWV5Ofn8/TTT5OTk0OnTp2YNWsW0dF1b2inkvxyO+//WTNIt7k8DZbndXm8hPkfWUE3G3Sc1yaG3s3C+b+ZWxndLYFzWkTw68b9dE0KbfA4o16rtSjgcHtxun2M7hhO0Kq3ah40miYCTserZOU6PBUmDa8tLng9kiscGC0TuFYXiRHojl+haT/ofC3Mf7km/crrkeMPT52qxuetv/R4TDt5QK8YX/9xATGyYnPwIi01AlJ9OMrBYJV2zHgAxs6C4ETocp2sJlUb6oEMHi4fDzMflkHAkOdk383fw7IPJOXj+1tEfKnGL0zSLNpeIZPfdV9JKkr+Nhj4BOQkiFnr3iWSEhPdVgZBe5fAjIckisfrkVSSYW/KPoqTRpXDw29bcrmwXcwRS2jnlNqICmw4dDg8wES5vUbQSCuorBVqvCa9gDd72TB/Ourg4DOSiUT6R8L1P8CWqRJeHpos6VHfja39PQtPFe+A72+V77RfmIiwIObfBTskvB2kf66eUH//KkqTsPWI5iKImPwhvptEwOXvgJ63S7SP2wEtL4KvRtc9R0mGeBJ0HgM/39XgZ8L+deJz0OoiuRfMf0n6QjXz/yfeW+X7odtNEmp/+MC8OF1Enc7XiW+O3iTvby9p+H0d5eK3034kfD60RkABEY8v/1CEV59Pog2rV5MPx+OqfeyhJPaU1edDKdsHy8bDkGclighk8lOcXvd4t0MiAoe/D+b6B7EKheLkUV2+2eZqODrhTMFXniuG8OX70b4cWXusZwkRb5hZj8vzwu2AX+4V8V7TSSRmRHPIWNLg+bEXy/26IRzldUUSTasRIIrSILo9ZNbvyyLeb2tkoaChyJ3m58vzxBxY/xgxpoNci71EbAJ2/1F7e9ZK6HsfLH2v5rXiPfLMDIqXZ3V99/Z+D5JtMxObMgRD2pw6mz3J5/BHZt1J6oPntaC40slP67IZf11X/tyeh91Ve7+UCH9C/Yz838y9fDymG4t3FdSJEp6bqZF84VtYpt9Z+w3MgWQPeJW3f6i9IHV1j8S/HNt0SQyhaYQ/ewpqi5wmvY4HzmtB4D8wI44IMPP+tV246qNlFFTUXEuQ1cDHY7oSHfzPq2KdSgoqnHVEnGoW7CygoMJ5QlKsdu7cic/no3Xr1vVub926NcXFxeTn5wMwePBgHnrooYPb09PTa+3/4Ycf0rJlS1555RUAWrZsyaZNm3jhhReO2A6v18uECRMIDJSFzuuvv565c+cePO6KK66otf9nn31GZGQkW7ZsoV27dkd/wY2QszIiB+Duu+9m7969OBwOli9fTs+ePU91k+rg8vhIPyRSZlduOT0byPE0G/T4mQzENXCzah8fjNGgw9+sJ6uoim5JoXRoEszdX61lxsYcksL9MTVQPvCCtjHM25538PdLOsTxw5osmodqGPK3HNKIIDjnYdj+q0wOi9IajhDZ8rOkU/z6GEy/T9I1Zj0hUSsDHxdPimpMf2FQVt+qQc87oTK3tsByKG577Yox+9ZIGkVDpAwUY2WQiVn+NhGspj9Q9z0q82US2/NOSfv45R5ZyTcFyGRr7n9qT65BPq/vbgKPXYxkzUESLdH7Hlmx+vMlmXzPflImxR/2l3PsmS9RT8M/kFSVrjeKsNNQeobihFDucOPzwc7cCjomhDS4X4nNdTBNsT5GdU3gl/U16ZE6TYz6qrm9ix/madfXFSoq8+GX+6HbLeI1kLMJfr6n7vescJdEwlz3vUSDdLmxZluHA8JNNU26w57DTBsPJX0RxHaWdMnLP5RBefEe+Q4m9hZRKK6TRNY1xOYfRCwJjGl4n8BY8HhknxUf1hZxQCJ5pj8Ave6S/taQoLJrDiT1FRHZUSb3qZYXNfy+TQdIpOAv99UVYnbPFbG56UBJtQpLqUmpqo/D07FAJiAdrpR74eFsnFLjQeT1wJaf6u5TTdq8A+XTFQrFqabSKUKO/QwWcryl+/DtX4+WvhDNUYb2/a11x3r2Eonc7HNvzWuOchkvZa2Q54ujQha+GiJnowj4DZEyALJW1fweEC3jJUepGB/vnC3pVfXdf3V6EfbXTJRnSEI9cwBLiOwz7wUY9kbdyBxrKFz6dk3UjiVIxKFD8bhEzBn4eO3XZ/8bfF58V0+B5ueJuAUi7gx7k5LwTtzw7V4y+72IL3VI7VMmn4P94ndZleM7uMgaZDXw7KVtuKBdLJd3acLD57fk00VpjL+uK4NaRqHXafib9Iztm8yLI9rz7C9bcLi9PP3TJl4f3ZGb+ibTIjqAzgkhPHNJG8JDg3hxTzOyrpqDs/NYaDYY37nPYL95Po8scLH/kGpZg1tFcceAZkeMyNE0jYQwPz66viujuzXBYpR9+zQL5/txfWgW9c+NiFOjAvn57n58PKYr953bnPev7cKv9/andWzjT/09UloaiJn0ieRo7XS7dWvY0wlg+/btdO/evdZrPXo04CF4CMnJyQdFHIDY2Fjy8mrmpTt37uTqq68mJSWFoKAgkpOTAcjIOEL64hnCWRmR01gw6XWkRASwO1+EgnZNQgj3NzF52d465mRmo45ym4vnL2/Pg1PXUVJV06ljgy08MrQl5TYXny9Ox+HycvfgVB6aWmM2+vnidF4c0Z7Hv9+Ay1PTYVtEB3BF1ybcPklEjCahVno3C+elWdtIDYnGE9YMfeEu6Hg17Jorng/+EeLiv2d+wxfX4UqZIFXk1n7dUS4T0iHPikkpyIpMbMe6JskAUW1k8lZNQLREsOz6HWK7yArJ9l/rHmcrEo+P6oiEfatFhApuItV7DsUSIuG7h5rClu2XqJ+KPOolZ6P43oCs2K/7CtqNkMHL4jfrP6YiVyaMwQkyMU6bJxPO35+uu2JTVSTpI5ePh8kjaqoe+IVLOk3uZghJqP99FMedMH8TOg125lWQEOpHVKCZvMMiczQN7j23Bfh83D+kOW/O2Vlr+0XtYzAbdOzMqxEGB7SIYslu+X77m/REe3LkO3E4eqN8l03+kvqX1E8GyyM/k+ppOr2sFK77Sr6bXpf00dBkKNgO22aIIFotCAx4DKJaHaj8VFL/RZuDoNUlIjrMO2Q1Zel74glzzRT5PmetbPiD8zhlAtDt5oYFn45XyeDc7ZR2NnSeot2yX0P4fLJyWn1f2rdKxKfIlnUFL4NFhNjdcxuOBlzzhYiurS+B4jSJdjpcZIIDn3NTSD1fDJZ9PhG6Bj4Bf/z3yGH9IH87/yNUb7CG/HXov0KhOClUVEfkHMEvpDHjK92Htnsu2qI34JK3axbt6qNkb10vNVsJ6EzyQMzbImMinb7++2xkS4moCUmqSduvxhwohSe+ulJ+1xulwlRokhgXD3tDxJKFr4mp8ZxnJKUJICQRLn5NInVcVTLGunIS5GyB1Z9KgYwWF0C7K2RhpNm50s6bf5dnUPEefIl9REjS9CLC5G/FW1UCIz5DS/sD3ZqJ4HXhbTcKb+tL8epM6JtfgG/NJAxV+bhaX06pLoLscj1tL3wV3YHqXF5TAOX6MGZneHjiolbYrVayB79N6OAyjK5yPKZAKg2hlGoBPDksiocvaIXT7SHAYiQ60IzhwILsjX2Suah9DG6vjxdHtMPrk4WhMH8TxVUuHjm/JZ8t3kOlw828bXncPiCFcQNTcXm9FJQ7mLR0L26vmX2mBAKHvITJ6EUzWjEDr46OI6vERlGlk6YR/kQGmAn9i4wAgECLkUCLkUeHtmLcwFT5U1iNhBzFsUdLXIiVuBAr57U5wuJQIyToL6KV/kk005FITU1F0zS2bt3K5ZdfXmf71q1bCQ0NJTJSBFl//xNTGcxorH19mqbVSpu65JJLSEpK4uOPPyYuLg6v10u7du1wOk++h+DJRo3+TmMiAs08MrQFd0xeg6ZJ1M3Ls7bz9tWdmbMlh5gQPzSkYs5FHWIJ8jPy1E8beXFEe4oqnGQWV5ESGYDFoOP5GVt47AKJcvlx3T6u6BpfK5Ry0a4CdBp8dH03tu4vo8rppmdKOCa9jmd/2UxkoJnLO8czrEMc787byfBOcfRoFUel/n6Cdv4mFZe+OVAZoM1w2PANJPZp+OKi24iHRX1U5Er0is4gKyXpC+Hyj+T8hw4YQpMldDdjOVz1pQwEQhJFZLGXyWQsrkv9EyydQSauoybKQMBVBTMeltWVjdPEoNXrlod51xvF0+NQD4/4Lg2nS1RzaEnJ/C0y6LEVHzHnGVuxlEtO6AV6s6xWdRgN/uHiA3LoYKdsn0y8Dz1fVaFE7ty+QHw7jNa6AynFcSfUz8SwDnH8vD6b52dsYcLY7rz46zYWHSgdnhLhz0tXtMek17C7fPRPjaBzQgjrsuTv1695BAt2FPDUTzVpNnHBFm7qm8ydX4p/gNmoR3PUE2HW8w7ocoP4LHmcInxoOmjSU0RRR5kMlFtdAmN+ksi30izxjtEZ4OY54itjDZWfbjeJ79Jv/xJT4vn1eOGAHFOWWddbqkk3GQCv/1pWSy98WUq81kdSH0j7U1KMet4uZbqrv886PZz/f/L7ngVyzvp8dKqxlcj5GiIwVkTSdV/Jv899WsqUj/hE2rp2slS1azZEqqiUZDYs1IL01fiu8vlkrpD+/cvd4mdTjV+4eBTZiiAiFfreLSmamctFTGtI5Gp/Jfgd4m/Q9QZY9Un9+/Yap/q4QnGaUOmQZ/SZmFrlK8kAZ6VUmuo17q89aKBuae3wVAj1iLkxwMqPZXz3/a2SKl9NdDsRagx+cNm74oez+Qd5xrW8SBYbdswWn5nIVtDxanx+YVCYhtb1Bhn7jD5Q5dTnhaun4nNV1gj6BjNEtEJrPxqfrViqKIYkwdVT8KGBOQjNVSmCu6bD5/Xg849G63YLGC149BbcmhGH24PT7cMdFY0uxofLA5Y2NxDQcjjgo1wXxOqMUj74M412cYGM7v40bq8btwc6RoTQwSTTsJxSG1M3ZfH1ygzK7dl0aBLETX2bMnVlFsv2FHJLvxTOadGUyEALFqBhtxlBp9OIaaCCVHSQnqt7JnJe22g8Xh+hfqZa0TSxwVb+Nyqk3mM1IDbESmxI/ec+GiICzUQENm7PmpNNRICJc5pHsKCe9KpzmkcQEXBiqkOFh4dz3nnn8f777/PAAw/U8snJycnhyy+/ZMyYMWhHMOU+lJYtWzJz5sxar61ceYTFvqOgsLCQ7du38/HHH9O/f38AFi1a9I/O2ZhQQs5pTs+m4fz74ta8/vt23F4vGUWSNpAQ7s+Pa/fh9cElHWIJMBuwGnQMahXNnZPXEBtsISLAzLTV+8ivcPDC8HZMOVD2r8rpobiqbhjegp0FLNhZQPOoAP4zvC1er5fIABMfXNuF1XtL2J5bRpXdxeMXtMLl8fLSrG0s8zNw5wXvE1qZXyMyWIJlgtLyYnnQV+bXvTDjX6i2zgoRWiJbSVRN1mq46mtJNyjcJYKNzytlJQ8dSES1lnQkvREyl0BQjDzMV38OG7+VSgEtL5LSljozWMNklSV9oaSFlGVD77uh771ScWvVJzD1ejEzriahl6SDOKtkVak+YcZ0mF9FaFOJOGh1sWxrKOUrtoNMDnV6Kcc+eYRMytuPgtsXiXHs7nliCu2qkvduO0L8d6onuR6nDHoylspk8/wXIKGH/F0UJ4TIQDMPD22B0+3lty05ZBXbaBEdyHW9kogIMBHub2be9jy+XZWFw+1hQItIzmkRyer0Iq7ukcjCnQV0Sgjhlv4pFFY46JoUStc4CwGajeHtwvh+YyHFVU7cYam133jgk9B8iHwfNn4r35nU88SHadsvNdE7Xo+Ik2XZcP5/RbAJihdBpzRT0vrajZQolOg2Eu0F0ON2iR7JWFr7ffvcKybkaybVft0aKgPsKddKXwNJY2x+vrxfs8GAT+4Ja74Q74DpD0oknMcl3lM5G+V7HdlCvsulWRIxF9dZ+lHxnvr/CMl9RfxMPU8i8g5n4OMi1rS5HFpeADMelEg3nV4E22FvSlh8ZQGk/SHCUMwRcquj24HbJfepi1+Ta7rkHfkMMleCwSi+QdMfkPD8DVNg2fsiprUdIe+bej7sOqxiSlA89Lqjxh8H5H537rMw99na+6YMFBH9KAdRCoXixFJ1hqZWeUuz0NZ9jTb/xZoxz3XfSwS23lR/ZKHeWDutqcUFMiaJai0LDpumiTgT3x3GLZOxTUWOPHOCE+Rev+5NGUv2vBP63i8RMAaLPH+63gCdr8dnMIPbjs/rRZc2TwpRbJsp1T/Lsw+Ot7wXvkZB62updHgwoSPA6MNscGEMbYo+oQd55Xa+Xp5BZJCFBdvT2JlXiaZBVomNKzoncEPfSFpGS5qODjACVlND0yjrwf92aKJjTO8k5m7LY8bGbEZ1TSAu1Frr2JhgK7cNSOGKrvG4vT78TQYCLQY6NAnhnnNTCTsBZr0RAUpMaSwE+5l46YoOPD5tQy0x55zmEbx8RYcTWoL83XffpU+fPgwdOrRO+fH4+Pi/9Lc5lNtvv53XX3+dxx57jJtvvpl169YxYcIEgKMWgw4nNDSU8PBwPvroI2JjY8nIyODxxx//6wPPEJSQc5oT6m/i+l5JXNAuhiqnhxdHtOflWdvYkVsjBLwxZyffr93HhBu7c0XXeJpHB/DuH7vYllNG86hA/m9EO1Ii/Plhbc1K8d7CSppHBdRK46hmf6kdf5OB4konu/MraRYVgM3lpk+zCGZtyWX13mLCA0xc3SORUD8Te1xJBPu21hgu5W0VMWLuc5L689uT4ikD8lDvcYesIJv8awsk1WiamKaO/ExWpctz4Odxsq3DlTJRmv3v+o3nCnfLRNQcAIFxkrNcuEeihJoNElFEZ5LS4Osmwc450O8+CZEtbCKTy2UfyOp/8/Mgub+snudtlVDeDlfK5GnlZzIZ7HgNrPuybjt63yUr/9V0vl6M/oxWmbzOq+fG12ywDGTiOom/SXlOzbZl78O26RLdkL8VRn0upZnNAWDyg2u/FVPB6hSR4nSJOshYJtXCRk+qXd1BcdxJDPPn1v5NGdWtCdFBFiYsSefnddl8ekM3Hv52Pav21ngI7M6v5JcN+3nnqk6kFVTx5pyd6DTomhTKpa2DGRKaS+ji9zAW7eCZiNbcde0dLCwMwmsNwtXpBozrJsr3sdWFEllTHTIOsH2mpA+NnlRbUAFJt9IZxBj4qq+lTLjeLIPw9V/B1VNqiyA/3inCT/dbxBPH6CcRYm6HRJkc7s3S+TpY+n7t91zyHgx/F/74PzFf9rpFkDnvP5CzWdKbcjZKOXGDFdoOl2gYTS8C1Y7f5DyV+bI6+sNtdT/8pL5iirz0fYmqi2ot3gf2UlkBHviECLM566H7rZLSVJ2u6PVIuPy2GZIG1ec+WPCKTEKu/a7+sH6A856Tv0FANHxzTY2Qag6Ei147YEzeQlK4ut5Ys9q8+gu44iP5O1z2rohky8fLZ9ZuhIg8h6dFWkOg+01Sqnf7DLCXy98+tKmKxlEoTiPOxIgcl8uFoWAn2p//V3vDjl+h47USTbnknboHdrtZ/L2soXIPbD9KRBhHmUSS9hon93m9AUxB+LrdDGhoXq9EFFtD8SX0hqoivD4vdrsTn6an3OMDn5vQUAt4vegNeowBkWjFe2HJ2w1eh74sk+ighiNJAsxGkiMCeGDqulrrc23jgrimZyIxRzj2SMSHWhnZLYERXZqg0zU8WbUY9cSH+tV6LcrY+MtmK44PcSFW3rm6MwUVTsrtLgItRiICTCdUxAFo3rw5q1at4plnnmH06NEUFRURExPD8OHDeeaZZwgLq9+7tT6aNm3Kd999x0MPPcRbb71F7969efLJJ7nzzjsxm/+esKjT6fjmm2+49957adeuHS1btuTtt99m4MCBf+t8jQ3Nd7QORgoAysrKCA4OprS0lKCgk2ug5XR5+GZlJk//vLne7U9c2Ipmkf5YD5gee3ywKr2I9//czQXtounbLIIbPpcQtiahVp67tC33fbPuYE43iLHqy1e0Z/qGbBbsKMDrgwEtInnsgpZc/fFySm21I3lu7Z9CldPNlc2hw89DRZjR6eGabyWSxRIiokZosqy4G8wyMbKXiGixoJ60jY7XiCfOotdk30vehvF9ZSIU1wVanC/mv/UR3Q4G/VsiZvathOAkmSBuP8Rbo9tNMnFdO7nmtYBoWQVqP0rauXGqGJg26S6TO49LIoE2/wA7f5PJ39VTZMK1a64MHirzZeW8912y6vTnSzLxHfQvOXb1BLn+W/6APX/CgldlP4NF3rfFUJnkRbWsf1AEElWwa66sgHW9UbxwFr8pA6WRn0mEktsu++2cLSbOIBEPN8+Rsp/lOXL9etORTWZPc05lX2yIrOIqvl2VRXGVg5IqN3EhFlpEB/LgIX5U1XRPDuXxC1uRX+5g7rY8flmfTbeEIN7qlEX4r7fXjvTSNCov/4LKkJZEVO5El79NvpOBsWIWXh/trhDh8NDvOcAFL8KSd+V7PfhpyFwmJcEzl0u0mjW0rjhpDRVxNrazCJx//Ee8C8z+8j2uZvQXIixVt90vDK79Xqq01ecfM+IjuYZvbxDhddCTIh4ZTCL0/HBH7f07XwdNz4FFb4q/gjlIRNLEXhKa77ZDeHMY8aFUpdJ0koK4eiKc/7wIMrGd4ZPB9X9mADfNlmop1jCJYkodIkLLzt/lHhSSJGlZQXHS56eOqXsOnR5umQsrP5VInNETpW+CiDGjvqgdcWMvFSHIEgq6xleD4HTsiwrFycbr9ZHyL0kbuKh9DO9f2/Wkt+FE9EVfcQbazIdkTHEoBjNc+SUENRFPv6XvSNRnYCz0f1DunV43oJdFNWcl/PECbP1RxiDdbsLX/HywBOHAhCcgHrshCLO7DIPBiN7kh8Hix2+bc7j7q7W4vbWnK7HBFr4f14fY6hSiinyYOKxm4fBwrv1Onl9HoNLhJqfUzp878sgrc9C7WTjJ4f5EBJr/UXlshUJRPy+88ALjx48nM1MVafk7qLtSI8Lu9vLzIRVtDufHdfsY2yeZqfPTuKZnIp0TQ/h5fTYZRVV8tGAPscEW3ryyE/+dvoWsYhuv/76DD6/vyubsUlbvLSY+xEr/5pFMXr6XP7fXhO61jAnkxZlb64g4AB8vTOPzG7vz6oIdvDH0fcKnjxWRY97zMPJzWdn+7YDpb3xXEVnS/pRJY8erxCx02QeSPuEXDt3GykTpl/tk0lSRJ347LQ6sRGevEZGioWiefg+AXyh8fqFMYs//rxznHwmp54oZatsRMOmy2sdV5Mqkbdt0uHKyRDWAeOF0v1VM9wwWaaPOKNf466OSUpHQQyadMe2lvf5RYn46aoJMTt02CRNuMVQiZOxlYA2Hm2bJv8uyJQ3m2xsl5WvrLw1/CXbNEe+hJW/LtVUbQtuKZcLe7gpZAYvrUlvsKs2SffbMl2igkgxJ4Rj4hEws/SPE2yN7nURkhCZBq2Gyj8mv3qYoauN0e/l4QRo5ZXZGdGlCamQA+0ttTFpWO5rDatTzyqgObM8p575v1lHpcNM3NYKPru9GtC+f8O+vqpuu5/PhP+s+/C9+Xb4nTbrDxW/A4Sukh5I2T4Saw4Uc/0hZEd2zQL7Dna8TAeX3pyUl8vz/1hVymg6ATtdIdMyCl6UvJ/eXPrbq8xrDca9H+ofHKf1l5AQxU65PxAGpVtf/ITHMvOg1+GmciLxDnpF+NGqihNBv/kEiZtZOlkiXXuMklQpNooNcVXDtNPmuej0ScaTp5d5RbZBcug+CEyXU/kiU75coninXStTOmi/k2kd/Iee2Fcnfx1kBC1+v/xxeD2z6Xvr2uU+LkFRNy4vkHIeKqCrtUaFo9FQdEoVzRpkdu+1i+NttrJjOazpJi9ryo6ThDn1BxmipQ2TcpjdCYDws/xTmHhj/tb4Eet+Dr/m5kNwHEnrgMQbhMQVgDgilut6qJN3XpKbnlNp55ufNdUQckOjx7TnlNUJOQKQsVEyqa8pKaFOIbvuXl+pvNtAsKoBmUQF/ua9CoTh23n//fbp37054eDiLFy/mlVde4e677z7VzWq0KCGnEaEdVor4cHSaRvPoQKlS9d0GwvxMTBjbnfVZJUxatpfPFqdzfa8kptzeix25FRh0Glv3l7E2o4THhrbkqZ83M2FJOoc/L/s0C+fjhQ1UJQA2ZJWQW+nluS1RPH/TIgK2fI2uYDsU7MR32XtoJRmSNpC/VUpxX/I2zP6XCCDmQBj6okwui3bL6vWhK/wgE8Wrv6qJqpn/kqzk//p4jT+O0Q8GPSFiyheXySSr5YUy8Tz/eTn/tunSjtIsMdeb+XDdkpled02ud++7RdD4478yIdMbxa9n9Bey+l+yVybCIz8TQSUkUTwyyrPh27FwyZuQPhXWTqoxRm52rkQcOcvFpLkkAxYecr0el0yOG8LoB54D5yrYWWMIDWKGfNl70HoYzDtsgq/pZDD2w+01r5Xtg5/vhv4PywBtwrDa/iNz/yNiXIuhR26TAoCCCgdTVmVid3n5bXMu46/rQpDVgP6wvN9/X9yazxbtYU1GycHXpm/Yzx/b8lhzU3j9ValAxI7q6kRZK6Uam/kIK67moLqluAOiZKBdnZaYvQYuekWMvoe9KU6Gmh4SekPmAV+cpgMknfDrq2rSg3bNlbSnq76SFK0VH8mgftccSY3aMFVExcxl9XtkVZO3VdppDRXB01YsK7x+4eKHpTdCfpqkKF1+ifSzojQRiDWdeGg5K6Rv97wTpt8vIjGI2NvzDvG0+v0pEU5m/0tEV72xrgknyE02ooVEsrW8UERmRzks/1B+QCLZrjzQp+tLuaqmYIcIZKs/k74JEjFoMItw3Iij4RQKRV2qDkQ3W416qs4QIcdrLxfvim3TxUsQZDzQ7WZJ9f71McjdAimD5T5etAemjRVvPoMPmp+HzyxpU96AONyRbbFYRSQx8NeTEIfbQ25Zw4UlNmaVMrDlIeml8d3kuTTr8ZqozJYXwdD/kyhKhUJxStm5cyfPP/88RUVFJCYm8tBDD/HEE0+c6mY1WpSQ04gItBi5vlcSy/cU1bv96u6JvDlnB/EhfnxwbRdcHh+ZxTZCrEb+fVFrwgJMrM8qweX28sCUdTjcNRUCWkQHkhzuz+JdhbXO6WfSExFgPmKhJbfXh8WoIyk6DAtV6IrSJMJj23S0vM3QZji+favQyvaLWZ3eKA/52U/Kqn6/B2TFfcWH9b+Bo0y8M4Y8JykO+9ZIiePLP5RVf5dNwnaXj5dVl/L9cpymg9aXiRHw1p9rzrdrjnhoXPaeeFscjt4IsZ0kHWzmwzWve1yyClWwQ8qjT39AJot6k5RBz9siniOjJ0OvuyTtaeVhlWZ2zxWPj07XS9Wa2I61t2+fCV3GiLlrfXQ4pIqQJVjEmWp0RohoKZEEh5cDTR0i0T/1seQtWS276H9Sonr5eIk48Hlh2s1w9yoIa1r/sYpa2F01fWpHbgUGDa7qkcjMTeJ3FBFgwmLS1xJxqqlyeqh0eg6uTP4lm78T8WXDN/Vv73hV7e99QJR856sj5KrJO1BRbdO3kgo08nMY9rp8F1Z9Ksa7391cu6IISETcr4/CBf+T1MDWw8TTyhIEe5eKwfHiN2UQ3RDVIs6oCeKJMPwDKMsSgbFaZG3STSLHdv4u5svL3hfTcGtojYB0+YciNB0qrDgrpfTsuU9DyiARPAt2SsRb5+sl5fJw2o2UCUtIkrxX7ibYs7Bmu9EKl74jqWmD/iX9fu/i+q+tSQ/ZtvN3OV+7EXItP46DC/9Xt+8rFIpGTeUB8SbYajxjzI61qkK0b66We2c1LhssfRfOeVgMjHvcCuUF8E5n6DpWIl9+vBNfvwdh2NvYjKH4+VnRceyTDqNeR5DVQJmt/oqFzSIPK5xhCZLnQ1wXWTDTGWU8ag48xndWKBQngjfeeIM33njjVDfjjEEJOY2M7k3D6J4cysr02pEkbeOC6JUSRp/UcFalF+H1+Qiyiuv9ztwKxn21li6JIdx7bnPm78jnsk5xTF2VdfD4t//YydMXt+HNKzsxadleEkOtXNkjEZ0mOcNf3NSDTxam1Vv67vy2MVzSMY5QXymm72+UVfZq7KWQMhit+fn4DGY0t0MmNqWZcM1UGRxkrRTfi4aEnPiuIoDsnCMmvx6XrPrMfU6iZwY+ARMvFgGm2aCa4/K2ysTs0Mnsodsylkm0wZ75Na9HtRYRpOuNtSNlDiVno6Ro3DRbTEhdNhj8lPh8eJzww61w4wz4tIFc7P3rJSXL4CcDjcDYGvEpf5ucM7l/zepXNc3PE4GpPBtiOkgE06F0uFKiEw4vBxqaLJEJDeFxSSTOd2Nlgjv0/6RENcjkN2ulEnKOAn+zgT7NwlmyW8TQb1Zk8PGYbni8Ps5rE83vW3LpmBDC0t2FDZ4jw+FPuCW4rokwSJTKoSbCBTvAFCieTEvfq71vk+7QfKhEgBXulMEsPhFxDh2Qg4iflbniEdX0HEkP9AuVtMKm/SWi5vDInmrspSJIrp0kFUR2zZFIldGTwOeWCJfUwZJaVJZV9/hOV4vYkvYnXP2NiDfV6YLVZK0S8eOiV0RI2fyDVJ6qyBERxy9crq2h6JhlH8A1U2DOc/L7+q8lLa3vfRLt5ygTQbbzddD9Nrnekgx4vxcM/jf0HCfn1psgMFo+64gWkobY47b6hRyTv3x+a76Acx6R+8L2mZJKBjKxUCgUZxSVByJygq3GM8fsuCyz7jOjmhWfwNhfwS9S7pcAtiJ8F70GbjtuczjGwDD+SXJ2VKCZW/ul8NrvO+psCzQb6JAQUv+BQbFA7D94Z4VCoTj9UUJOIyM6yMK713Rhzd5iJi3bi88H57WJJszfxJb9ZTg9XvLKHbw8azuFlU5igy3c1K8pD5/fkldnb+feb9ZyVfdExg1MZXN2GZuzJUrD54MXf93GR9d35d7BqRRXubh5wsqDK0x+Jj0Pn9+S2BArU1bWCAUjusRTZnPx9tydvNDfROShIk5sJxjwqFRrqizgYIJJm8tkxeSrK+H8/wNziEwUI1vWVF2qRtNEJClJhwGPyCTqUA+ZwU/BH8/Lvz0uWeHX6cWjIqK5TNoaYvMPMgmuFnKC4iTN68c7JWS4tJ6JZzXZqyW9ZN9quZ4Bj8EtcyT9ImulGAofOuk+nKxVsPtPiZS57D3xramOwpn9tJijuu4TrxJNBx2vlsoOKz+VSIFmgyW9q5rQZPHxmPtfSU3J2SDlk6u9fX68E66bJtV94rtI6fTtM2oqY+mN8iVY84VMmCOa1wzebCUNX4fiIMFWI09e3JrL31uC0+Mlu9TO/jIbXh88OKQ5o7o2YX1mCWWO+lcWAd5bWcEHw97BOO2Gw8yOdXiHvYVuyZs1r/l8sPJjSQFqcYGk79jLRPAzB0kaltsB4S1ESFl3WKnw+K4SrbJvtYiLcZ0ljW7failznTpEztN0QMMX3eFKSFsg+351ZU3UTmkmnPOolAJf+xVc+JL0h1mP1/jpND1HBMsVHx/wVTDVX80NJBWpIleMjC99R8TZ6vMEN2l4ogEizBgsIppWf27T75e+P3am+OB4XHI/yNss5sUjP5NjZv9bPLJ6jZOS5PvXiwCUNh9+e0LEn4tfF6+iygMid0RzSbHc+btEA27+vnZ7zEHiwaNQKM4oqtOpAi0G8sobTgdqVOzf2PA2ewngg/e6H3zJ1/IibJoVv4gEjMfh7Q16HVf1SCC9sJJpa2oqr0YGmPnsxu7EVfvjKBQKxVmIEnIaIdFBFi5sH0vr2CA+X5LOVysySMuv4Ls7+vDVigy+W10jQOwvtfPCjK3cNSiV89tEM3tLLh8tSGNYhxgeOq8FVpOe1XuLCfM30SkxFKtBR4nNxY0TVtaaR1Y5Pfxn+ha+uKkHm/eVYjLouLZnIi2iJVz16WFtiHAftiI+4FHxZDm8TPiWnyTVIHUIvvBmaF43rP5cvGw2TZMJldshfjf9HwJTgFSpqS4XfCjhqSJaVLN9JnQeI+fTGxtOJwLxmonvJu0MTZYKWzqDTAx9PvGycDcwGLOESBSN1y1tzlop6WLWEFnttwTW9q85nIBo2LsIetwsn1GP2yRM2RQklYB2/i5pWUkHqnX9eIdEVzQ9RybcFblSqcdRLkJSZCv4arS8vnuOnC+hhxjYlmXLhNkcCAk9IX0BmINFBLMVw/KPxH+lmtUTZYJeLZAl9mr4M1TUIjncj4k3defD+WnsK7Hh9sDD367nqWFtmLQsnWcuaYNe0zFpaf3RI1EhAXxdGMyF184lfP2H6Aq24Ylqi7fnXeR6gmjS4iIRWqo755qJ0PseiajS9PLdtRXL/+c8I3/Hha/DyE+hIlvERxCxYcBjUkmpOtpm8/fiP3XDdBFMNk8ToS+6rZgOz3yoRqyoJiRRInq+PiDimAKgz90QlAAfD6wRdlZ9Kn31+p/k+xfXRYwpP7ugZh9bUcOmyCA+DMnniDjrKKvxiqrMl+ttCHOgiJGXfyzVqAp3Sz/N2woTLq6JfrKGQqsDaWBbfoKWF4jHVtcbIHu9iFVTb5DPs7pfr50skVGXvCV9ylEqpsoB0dD5WtjyQ+00R6OfRAcFHaG9CoWiUeJwi5DjbzZgK2ogirER4XVWoQtLaXgHo1XGWbYD44fQpvgSeuIXfHwjDiMDLTxzSVvuGpRKVrGNQIuBmGALMUEW8e9RKBSKsxQl5DRSfD4fE5ekM3FJOgBmgw6jQcf3a+qPIvls0R6m3t4bm9NN+yYhFFe68LcYeGXWNpweH2U2F+mFVdzQJ4mSKleDnjiTlqVz37nNWZtZwptzd5JZZGPGvf34ankGl6To6RWSKGkJocmyKn+4iFPNmi/w3DJXvFXH95XIALdTVvB73yMrPVmrYM6zYk56+ASyGnuZmIZWR5asniARJYOfgh2/QqdraypQHU7rS2HtFzIxLs+RFf6AaDH+DYyVKJjVE+oeZ7TKxLEko+a1kgxJedrxm0QNXfkltBkOm76re3xwgkyenRWw+UdZ8a+ORBjwmGyv/v3Q1fwNUyTywuuFVROk/HPn6+Rc816QCXFkSymvntTvwCS1RIye+90v1YMWvS4loQNiRORpNliifw6N7infL6WXQSI0lEHgMaDx6cI9NI30595zm/P49xuocLiJCjSzaV8Zo8Yv44EhzbmhdxITDxNzmkb4M7ZvMj6gSq/DOOQ1KitLybXpyc8u4dxEG77kfmi3L5T+kLUKYtqJQbE5sKZaSMZSMaB028XjZfh4WDNZzLt73C4iSEQL+ObauilT5zwCS96p/b3b/YeIMJe9X9vwWNNECM1eKxFwQ1+A0BT5fk24qK6nTuEuWPq2+MpMHiHf0YteFR8mr1sEKP/Ihs2RQ5Mk9dAvTPYLjpeKVGXZ4s1jDa1rXg7Q/Rbxc+h+i3wms/9d/3t0vRHWT5F/VxXJvaPH7TDrXxLFFpkqPjf719U+LneziGg/3CZtGfp/IuKGJooolr9V7jGhTUVcDWoi0XUKheKMwnnAdzDAbDgjPHI05wH/Qf+I+sdgna+TFHGDWbwQ+z+M7gSlYQdZjQRZjaREqmpSCoVCUY0aTTZS3F4fmcW1J2GFFY46Fadq9veSU2pnUKtoEsKsrNpbzPrMElbtLam1n7/JwJrDXjuUvYVVbN5fxvt/ij9Lq5hAzAYdo7s1weuDsvNeJ+i70TLRKsls8DzYS8DjwueyyYTst3+Jr0RYM0mxuPhVSRspzZIIkYGPywTscPRGmaBVR48AzHxEIk+6jJFS3bGd6k6+/MKhyw3wyeDa1WsqcqVs98LX4PZFEgWQtaJmu9FPUqEWv1W3LRnLJfKl41WS3jXwCVmp2v1HzT6hyWIm++0N8vu6L6X6w63zZHU/NAXWfN7w57Z2slS+aT9SJtf718IPd0qFoAGPiTi09kuZkN44UyIQfD5Jb7vwFan25fVIdFNoknzWmiZG0s5ySXOxhsjnfu6z4mGi/DyOGj+TnnNaRvL0T5vpnhzGjtwKAJamFXJx+1hmbNzPG3N2MvnmHvRNjWDW5hwq7G76pEYQ4W/iqo+WkRjmx7COsYRYTbSMDqB1cCmdFz2KNn2BvImmQatLRPws3ScDbEuQRKpUpxtVU5En3kfXTgOjRSrCbZsu4t3hXkqWEImUm/Ns3Qsr3CVeMD1uE5Enqq18vyuLIaQJ3PqHfDdXTxABxNvAJGbzD9DyYhGQds0WQePGGRKBVrxX0p3qe3+DRcyTv75KxNsrJ4twWt23/3wRLh8Pv9xXI+qCiKkdrpLrtoaKCHvVV2IIXpFX83m2u0L6U7WHTYuh4HHDZ+fL/eGcR0Rs0hlh7CwxWS7bJ75CrS8Rj6CkvtD2ckCrET+D4+UndUj9n4dCoThjqC4g4X+GCDnYi8Uvb/gH8Mv9cs+rpvn50P5KWfQZ/QU+/0jcxoDjkk6lUCgUiqNDCTmNFKNex8CWUczZKpMRl8dbb2nyDk2CuaVfU/Q6HXq9RnKEP3aXh4vbx/LOH3XTGLKKbaRGBbBxXz1mq0BqVAD7im0YdBoXto/h3nOb838ztpJVYmNk1wSiWnWBG/7AuvZTjNFtGr6AoHj0mcukQtKAxyUixFlZkyb1zTWSzrFngUTWaDqZjC58DQp3iODT/RaJBmjSXarmbPy25vw5G0B3o5TQPudh8bbYNE1W5FtdLNE4jrL6SxCDCDZeNwx/X1IjstdCWIpUilr4Wk2J40OJaS8pI3OfE2HJaIXz/iuGqQU7RagKjJFy5iUZIiZ1vl5W6TWdRAOFJtVvdFuNvbgmKkDTwZ8vi3izZmLdffO3Skpat7HQ+16JZMjbIlE7Rj9pX8EOCGkqk/P0hdLW+G6gN0v7/EIbbouiDpqmcW6raN6btwuP14fZoMPh9vLpoj28MrIDsSEWpq7MxO728uDU9fRvHoG/2cAXS9JJK6gEoLDSyQPntWDMZyuYc3sbLLNuQztUTPT5xCNGbxLxYfGbMOZnERsWvS4CZXVFpOx1Ermzfy0s+xCGvwcdRsnf/3CaniN9rSHWfw1XfALhzcRsfOM06HmbRMHYyyH1XEjqAzmbGj6HxyXCSWwnScHSGaV/tR8FtkJAg07XwbrJNcdYQsSzZsGrNVFwtiKY8ZD4PqX9Ceu/EXH18g/lnCV7pT9V+2U1Gyjm46O/AKO/9MvgJiIeBUTC7nnw8z1ybv9IEV4qi+S9m58PpRm4zB2wJZ2HozSHiIFPoG38Vsq352yE8BSJQPr9Wbj+exVxo1CchVSnVlmNelweHz6fr/Gn/uxfJxGeg56Q9FF7qVRAzFohY66hnfBZw3H6xWIOUdG7CoVCcTJRo81GzKCWkYT7myisdOL1QU6pnahA80GTvd7Nwrm6ewJP/bSZUpsIFnqdxnW9kogLrr/I8W+bcxh/XVd+Xp+N57DwHp0Gt5/TDL0ObjsnhfxyBztzyhnZLQHTgRKRe8t85JVHspkbuD08iIjgJvWbBvceJ6vY+dvE4HjYGyJwVHu1VKcfNRss6Vll+6EwTaJdolqDKQBf7ma0ZoMhMA7iOknJ7/ytknpkDRaRKH2RTHoTe0GfeyQdaek7MOlyScFqO6KuGSlA/wdh8dsyIesyRtqzb7VM7OoTcTSdTPg+O18EKRBz15kPSzTB9d9DyT4xE979h5RD7nWHlDFe8pYYoHYZI6v3yf1rV/46lORz4PenJJogoWdNbnp97FsNJivMeFCMmB1lUv3r96dq9knqKwJUm+EidP18L1z3vaSJJA8Qk9qAqIbfQ1GH+FAr397Rh/nb87i0Yxzfrs7C4/Xx0LfrGdAikv9c1g6LQU+Fw82vm3LqPYfZoKNjk2D8XMW1RZxD2fw93DhdfG2+vEL8Z678Uv6+C16RfZL6iIH3io9lID7zYYmuuXKyiDmHplbp9A0LmyCpj0XpEvGmaSK0LnlLInGq06hiO0lq0aLX6z9HZCvxner3AMx4oKa/azroepNE5HS/VfpG1iqJDtN0cm/IXntIWw0iRBanQ2Jf6TNFaVJSfOfvEqnnKBPR6/IPxUPI6z5QhW257KNpcM5jsORtMTzX6SVaaMBjEjE39CW4+mt8BbvxJvbkum/3k1GUz5c39yQyMkA8cBxlkj6Zs0H60KjPpV0KheKsw+GS+6DZqAPA4/Vh0DdiIcdglqjmjCXw091yXzX61dxbr/kWHFV49q3F3qED5lPdXoVCoTjBJCcnc//993P//fcft3MOHDiQTp068eabbx7zsbrj1grFSSc+1I9v7+hNv1RJfRk/P42Xr+hAoNmApsFt/VN4+NsNB0UcgMGtoujTLByXx8dH13fltVEdaR0beHC7w+1l8vK9fDymK5GBNY/liAAT713ThZkb95NRZGPiknSu+WQ5475ay7gv13DLF6u45YtVpET607NpGB6DH5d9mcXeYVPwNjnELNcchG/Ic+CoqKnStG+VTCgveFlWzK/6WtKNHBWSRpE6RCZaqz6BX+7FN/tJimwetlq74CnaCwXbZHL56bmw8TvwD4dpt4iIU03uFhEs9Cbx1fC6ZcW+1UXiHWIJlv2C4sXrw2WDjVMk9SlvqxgMJ/WF9leI6fChaBqM+BjWfVUj4hyK2y6pTjFtYc9CSVXqdYd4lKQvlAGRvfSAN8nP0Ha4CEaH4xcOyX0lSil/G3x7I1z7nVTUqW9/a2hNe3QGEabS/qy9z97FMP9lMa01BUDXsZL2dum7kLepxhxXcUwkhvkxsmsT7j23OW3jggD5M/+5PZ+Hvl1PsJ+RhhZqh3eKIyLAzCcjmxKtHUGo83nFm8XkL/3HUQqThkPGspp99i4Rr6Te40QAzdsq3+0VH8Ogf9U+X+ZyiappiJYXiEE3QIsLYd9aESYP9cLZv06iz5L61j1e0yQ6LihODL4PNdj2eaV/7/odfv+3CDSL35Ky41PH1BZxYjtKiP+oL6TP5G8TM+Of7hKTYleV9Kdqo68//iupaOEtpAz48g8lLawsG364Vfr26EkiTPV7UNKz8rfD/BfBEopWsR/f6i94+bKW/DCuDylRBzwaitLlHIP+LZ9l2T746BypHKcqvSkUZx0OtxejXsNwIDra3VCue2PB44a+94rIDSL0V99bo9tKlPKqT9GFJOBnUuvCCsWZxvjx4wkMDMTtrincUlFRgdFoZODAgbX2/fPPP9E0jd27dx/xnNX7lZSUnIAWn32oO28jJyUygPev7UJxlROP10eI1civ9/dnV14FazNLcHpqJll3DEjBz2Tgvm/WYj+wchQZaObZS9oyaVk6y9JkYuXy+IgMMDFhbHf2FUv55Cqnm29WZDCqWwJGnUaov6lOW8YNaEZkgBm9XscdA5txVfdEdhVXor/wM6IMFejcDrzmYAw7Z6DNeabmQI9LonZ+e7L+iwxrJt4u5fsB8DmrmLTTyG0ty9Bv+1mMSHfNkX13zwVXJVw5SYSi4gyIagUpA+HXx2WCefNsqTxjL4Of7hRh5qJXJHLGFCAT491/wJBnIaGXeItkrxH/HI8Tmg0SY9R9a0SUiWwpETJZ9UTqVJOxVExeL3pFfEhmPFh/RasV4yHqLUklWfmJlJTWdNDyIkmRmvlwzb752yQ1pHAnXP21iDT710lpZI9TopQOlnPW5LOpj71LZGA2aoJU+5r1hKSXdB0rk+/KfPldcUxYTQYSwgx8fmN39hRUsnhXAeEBZnokh2E2alzcPpbpG/bXOubxC1vRLcpH8N7ZRKx8VXxwGkLTyQB7zM+wfRZs+bl+IdFZIX/X1PMkda9wt3jphDcXwW7Fh5JiZwmByNYS6XV41Jk1VLxmvr5Sfm93hZTwro/fHoeb54gIuu5LGfjHdhIRJ2uVvL/bXv+xS9+FPvdKGtWQZ0XwORS/cLjkbUn9+3q0+Pys/QJaDWv4cyrNkr5+xSci8nhckkZ18asSZbRvNVTkiMH59PslVQogcwW4bWAJweAXSbLVBsHhsq08R/Yt3iMRf4eyZoJEFllDGm6TQqE443C4PZj0uoNCjsvjxWLUn+JW/QOqCqTi4OhJErmYsUzGR+1HQ8sLRbze/CM0G4TR0PB15pbZyS6xkV1iIyHUj9gQC5GB9UeFKxSKI2ArljG5vUwWoP0PFHo4QQwaNIiKigpWrVpFr16yKL9w4UJiYmJYvnw5drsdi0X68rx580hMTKRZs2YnrD2H4vP58Hg8GAwnR8pwOp2YTHXnvqcaFZFzBhBkNZIU7k9KZABhAWaahPrROTGEPfk1k7omoVZSowJ4/fcdB0UcgPxyBw9OXccdA5phMmhc3jme63olUlTp4tJ3F3PbpNXcMXk1D05dz/ydBdz7zVoCLAb8TDUPbbNBx50DmzGyWwJ6vXylogIttIgJ5NzWMTSJj8cU3RJDfAdMEUnowpJrX8C+VZIa0RDxncX4eMp1MOU6bP7xjG0D1vUTocNoSdE6lIxl8OUo2DFbRJeUwTDxEvHWSegOX18j4ovZXwSMXXPg+9tk5X/yCKm4s/1XqapTmglTrpd0kDbDxY9k9r/hxzuljHJRmkwuA6JlItgQgTEiiljDZPKdvrD+/UozwecWQ9qgOLhtvkxA/cLk+gsPU7rtJSK4ZK+TaIvAWEnvGPMzGKwi8HS+XibzR8JVJb5E22aIYJW7if9n7z6jo6q3Bow/09N7A0LvIB1FUCmiIGJXFAuIHcWKDeurchW9tmvvCnaxK6KCAiodpHfpNUB6n8mU98NOMplkJiSYCvu3VpbmnCn/CXOSOfvswi+T5GdTFOCkW1VJQkQQ/drEMnFYR64e0IqYMCuFRW4mDGnHZSc2x2aWY6ZPi2hOSLDSfv93xP10DRxaD+lbJQjiT8eRsGe59Crocp5vU+7y9iyTkqoTi6eTdbsEfr5PSqC6XiiBl+6XSnCi33gY8pAcI5HNJTvukvfh53vlOAB5PwaaSFdUIIHQokKZSnXzQpkAFxwtwdiy097Ky9wNzfpI6v6SNyVD76ynJYh51hQ5FuzZkn105mTJohnysGSUBWIJBrMV/npWegCNmyn3i2wB4YnSG8gaJsd2SRAH5Hh1FUkwdPbD0o8qfYc3gy5jR+Dn3L8i8D6l1DHJXuTGYjZiMsrvdKerkWfkmG1ShjrjLvk9OWoanPM/uRDw+RXyuzXtH2/Gjh+70vIY/fZiLnx9IRM+Xcl5ry3gineWsOcYGM+uVJ3K2gdfXguvngjvDoVX+8JX18n2WtKxY0eaNGnCvHnzSrfNmzeP888/n9atW7N48WKf7UOGDOGjjz6ib9++hIeHk5SUxBVXXMGhQ9LPdefOnQwZMgSA6OhoDAYD48aNA8DtdjNlyhRat25NcHAwPXr04KuvvvJ5fIPBwM8//0yfPn2w2WzMnz+fwYMHc9ttt3HnnXcSHR1NYmIi77zzDnl5eVxzzTWEh4fTrl07fv7Z2wPS5XJx3XXXlT5Xx44deekl3yE248aN44ILLuDJJ5+kadOmdOzY0e/P6N133yUqKorff5eL5evWrWPEiBGEhYWRmJjImDFjSE31Tv3Ly8tj7NixhIWF0aRJE55//vmj+Jfx0kDOMSoiyEKflt4o7ag+yUxbuMvvbe1ON6v3ZPLxtf0wGw38si6Fr1fsq9AjB8DtgenL93JZ3+b8NnEgP956Kr9NHMQdQ9sTF1bFCumknpKSW8Jpl+BL3+v8vJBm0OMKmXZjDsJ16t3YIpsQsXe+nGg17VVxUk+J/SukvOqPKXLi1WusXFXKTZGGxSarlBxVeJEueczUzfL47iLJjinKl7IRkMBIxg4pzfrpbnm8nlcGfs0nTwCjEfBIGYm1khGaO+dD3+sl68FZKGVif0/1n20R30kyjX59QDID/p4Kn10uJ/WrP5MglMHov/SqLJNFsnjKW/tl4BN2dVQSI4Iwm4ws35lOsMXIi5f15PUre/N/53UhyZRN5KL/em/853Mw9BEJbpTVbqhMFPv9MTyWUNkWUsl0sbAECbC4ndDvJunJlJ8qgcjfn5BSpt8ek0DGV9dIwKPnlTLprN9NEhRsfjKccIn04YlMlscIJL6THJ8hsfDtePjlfil96n6pNPeu7H77V0qfnaIC+PRSyajrfL4EK/PT4J/fZb0/3CrB109HyZUpfw2cQda89G3Y+KOUa70zWIKV+WkwYyJ8cok0J+8wQnp1ldS8tTtD+kblHZarYCs/hHeGSCNlf783yqrs+K6M0+4NlimlGhW7043VZCwdPFHkdh/hHg2cNVR+7+cehD/+K73DvrlBPmtEJsvnitAYjAEydlNz7Yz/+G92pPp+dvnnUC53fLGS9Dw/nzmUUhUVZEifqu1zfLdv+10yjAsyau2phwwZwty5c0u/nzt3LoMHD2bQoEGl2wsKCliyZAlDhgyhqKiIyZMns3r1ar777jt27txZGqxp3rw5X3/9NQCbN2/mwIEDpQGUKVOm8OGHH/Lmm2+yfv167rrrLq666ir++OMPn/VMmjSJp59+mo0bN9K9e3cApk2bRlxcHEuXLuW2227j5ptvZtSoUQwYMIAVK1YwbNgwxowZQ36+BJDdbjfJycl8+eWXbNiwgUcffZQHH3yQ6dOn+zzX77//zubNm5k9ezYzZsyo8LP573//y6RJk5g1axZDhw4lMzOT008/nV69erF8+XJ++eUXDh48yKWXXlp6n3vvvZc//viD77//nlmzZjFv3jxWrDj6i38ayDlGGQwGTu+UQLhNTjiaRAVX+GNa1oYD2XyydDdf/r2XJpHBbDscOINj44FsMECzqGCiQiwUFLlIzbVT5KriuM3IpjD2Bzw9LpcAAsD2eTj7XEvG6Bk4u1wCbU/HdfYLuC96T6ZA9bwK99gfMOQexPzt9bB3MZx0k0ybKX+SW1bzk7xX2Zv29O2bs/YrOOWOivcxGGUcd3Rr+eV4/mtwxv9JyVJsO+/tTrsbfntcAka7F8pjD7xX7l/6WAbpuxHdSj4Q/T5ZTkJ7XiUlXaOmylWu0Z9IhkFEUykDe3ugnGRjCBwganmKBJOy/GQ4/P6E9NrxuGWiVeZuSYX2p/Vg3xHp5e1eFHifOipRwRaiQix0bRrJLZ+s4LW5W4nxZJJsyfUtuSvIkEBep3Pgii8k0+qK6dCkF3wtgU9DfEdctnA8A24N/IQn3SiBl9i20OVCmfTWokwfm6J83zHzqf9IRs6az+GN/hJQcRbIeyjvEDjyJUvGn8hkydjpdzP89Zw0Ag5LlF5UwdFyHARKBe4/ARa/LicLQx6QbSnrJDOm2yj46R5oNQC+G+8bYPzreRk/Xj6A0qK/9PYpW/7kdsL3t0gg59AG2Za9T3rp7Fkqv1dCYqT31MqPfR+vIAOWvC2Zdf76AIEEiJNO8L8vkJyDkv325TUSnNo8s7SUVCnVODhcbixlAjmNPiPHYITTH/L93AOSYXzBGzJ04tyXIMJ/NnJarp2NB/xfCFqxK1MDOUpVVd7hikGcEtt+l/21ZMiQISxYsACn00lOTg4rV65k0KBBDBw4sDRTZ9GiRdjtdoYMGcK1117LiBEjaNOmDSeffDIvv/wyP//8M7m5uZhMJmJiYgBISEggKSmJyMhI7HY7Tz31FO+//z7Dhw+nTZs2jBs3jquuuoq33nrLZz1PPPEEZ555Jm3bti19rB49evDwww/Tvn17HnjgAYKCgoiLi+OGG26gffv2PProo6SlpbFmjUxGtlgsPP744/Tt25fWrVtz5ZVXcs0111QI5ISGhvLuu+/StWtXunbt6rPv/vvv53//+x9//PEHJ50kFyhfffVVevXqxVNPPUWnTp3o1asX77//PnPnzmXLli3k5uby3nvv8dxzzzF06FC6devGtGnTfHoQVZf2yDmGNYsK5oub+nP75yvZn1lAm/hQ1uz1P9q6U1IEi7ZLZsuh7EJaxISwfn+239u2jQ/D6fLw5KyNTF++F4fLTZjNzI0D23BFvxZVy8yJao7hnBfIO+V+dh3OJiwimm1ZQdzz5X6GtB9PeKiBVcvstIsxcVH/t+hl2k7QB2d5G6s6cmD7XFj8mpRefHm1t7lpiehW0gekpJTD45Er6SUnymu/hNPukRPAFR9C5h5ppHrKHXLbyGTZvv4b6dex/D2ZtvPDbRKgiWnrHZcO0t+jx+Vywp26BazhUpoS3Ry+uhbCmkh2Q3hTKV/Z9INMibIX/5wTT5AGrmGJcO5LUt4V10GmYbmLYM10WbvBINuGPgbvnxX4Z7x3GSR1l745P94uvYE8btjya5l/zKEw4r8SOArErLXsNS3P7uTNP7ZzZpdE5t1yAk1c+7EV/iPZGJd+JH1d/vwv5KVK8OD3x+WOoz+RJtkgWSjnvQKzH8W0fyUF45cS1P9WDIte9X2yE68HW7hMt8pPk/f1qRNh0H3ywcDtlPf24Ackuwzk//98VnragLxH134lX4MfkABsTFsZdb7qE+8xlXgCnPkEHNokfXk6DJdjLCRWmgjPvEcCRBe9A7/9Hxxc730tp94lx03aVtmWuhUueleaauYdlhT+jmfJqPHyx/reZZJtN/pT+Xll7YXkPrL+r6+vOI2rMEuOI5PVNxNtzedw3WzofB7MuENKF8vb9KM0AD33f3L8lc0INBjggjflGK6qnIPS1Hnbb95tW36RSXuXTA14kqSUaljsRS7fZseNPZCTkyKfXYb+H1hDpLQ0MlkCPD/egeeyj8mN7UV4gLvn2is/Ock7wn6lVLFC/+djVd7/LwwePJi8vDyWLVtGRkYGHTp0ID4+nkGDBnHNNddQWFjIvHnzaNOmDS1atODvv//mscceY/Xq1WRkZOAuzkzcvXs3Xbp08fscW7duJT8/nzPPPNNnu8PhoFevXj7b+vbtW+H+JZk5ACaTidjYWLp161a6LTFRPpOVlHgBvPbaa7z//vvs3r2bgoICHA4HPXv29Hncbt26+e2L8/zzz5OXl8fy5ctp08ZbYbJ69Wrmzp1LWFjFrOxt27aVPk+/fv1Kt8fExAQs26oKDeQcw4xGA12aRvD5DSeTXVhE9+RIrp26vMLtbGYjw7sm8fIcKRv6eV0KL17WM+Bo5BsHtuE/Mzbyw5r9pdty7c7i/jsu7jijPbZKGt+VsoTgjkgmLz8HJ0b+OZTGgLZxfLXKeyV65R64pHMQQb/c6jsdJ6m7NOnN3i/N9i58W048U7dIvXaHETJGOD9NyjW6jZIyrc7nSplWib+ek74y3S+DkS+CxwWOAkhZ5c1ScDmkN036dmkuPOh+mfrjL5Vx9WfyFdVSgjEel0yqKsyGA2vgn1+lwazJArPLNbI9uE4auF70npRInfW0vLaWp0C/W6S/idspJTE758t67P4Dc4BkTZRkPOUdlilgHUZA32slYGC2yVj3onxo1lcmg4UlQlx7OdFNWSsf2FqcHPg5VLUdyCzg8neWkJJdyJXdw2hBOsbfHpTm2SUSu8p7+psbvIGCyOZ4YtpiOPtZya4x22D+/yQjrM84gnJ2y7/fFdPhwGrpX9DyFFjzhZQOnXKnlAcNnyJ9niKbSUbYXy9IJkpcJ7j8c5nqFN3KG8Qpb/Hr0ntq00/SY+ayj+UYMVklCPPj7TDyeel9tW+5ZJx9N14agoNMhPrxDglmtugn06uKCiRout2bvkvKWojOgVUfy0S7bb9LELTsbcrau1xKuC79RAI3bre8zkCchRLYLRvI8Xjk5GXDDG8ZZXmWEJkWFtcBbpwnk922/S4ZfL2ukhOd6owg37vUN4hTYvdiyZTrVUnJplKqwZCpVcdQaVXmLvns8MOtcjEgLEF+Xxd/9jE4cvE4zUCs37tHh1gxGCrG3QFMRgORIZZaXLxSx5CgSkrZq7L/X2jXrh3JycnMnTuXjIwMBg0aBEDTpk1p3rw5CxcuZO7cuZx++unk5eUxfPhwhg8fzieffEJ8fDy7d+9m+PDhOByBM/Byc6UK5KeffqJZs2Y++2w23+SA0NDQCve3WHx/lxgMBp9thuKS+ZKg0ueff84999zD888/T//+/QkPD+fZZ59lyRLfQR/+ngvgtNNO46effmL69OlMmjTJ53Wce+65PPPMMxXu06RJE7Zu3er38f4NDeQcB+LCbcSF24gNszH5/K5M+XkT+Q4pg0qMsPG/y3qx/XAuvVtE8/euDHLtTn5Zd4DHz+vKf3/ZRF7xbUOsJqZc2I3IYItPEKes9+bv4PKTWtA8JkC/inLsTjefLtnFjDUHePqi7tw0qC3/HMplU4o3HTfZmuc7qhikX0zJVe/138DBtRKgiGgmwYd9K2Tyzra5kgL800Rp7HrpR1IqlFMmSOUukivwf0+VkeOuImg/XMojmp8s2RAxreW2fz0nwaDzX5EMm0Cy90qWwRdXybSeojKN/ezZkq3gT346HN4omQDZ+2DRK/LV8Zzi0pQoOblb9JqczCefKNkI/jTvJ68Jik8+jf4nDXU8G8583DtN6MAqCE2QiUmOguplF6gj2nIoh5TsQmxmI+e2C8L42yO+QRyQTJXfn5Dgy+xHwBpK3oVTCd33t7yXUtbL+/uEi+T9ZTBi+PA86Z0T0waa9pbHsedI0HPVJzDsP9BygPSUyU+XqWdp2yWYk1ncP2vXInmMvFQCKsyC3MOSXfbj7RKAKWEwSgBz2XvyfVQLCTzmHvJ9jOx98rqiWsKA23ynsZWIbQNJPST77qML4dwXJSstrr3vOPKyErrCghdljPmIZ6WEy1/A1WiW8qiyx2UJSwh0PhvWfu7/OU66EcLiva+v7zXQe2ylDT8DsudI/55Alr4tWUgh/k+UlFINR/lATqPPyCn53APyu6psOavZBtYwzGk7oIn/KTVxYTa/0xkBLundjDg/00+VUn6ExksGvb/ps22H1vpk2SFDhjBv3jwyMjK49957S7cPHDiQn3/+maVLl3LzzTezadMm0tLSePrpp2nevDkAy5f7XhQsyXBxlWnH0aVLF2w2G7t37y4NFNWmBQsWMGDAAG655ZbSbUcam17WSSedxK233spZZ52F2WzmnnvkM2zv3r35+uuvadWqld9pWm3btsVisbBkyRJatJAhHRkZGWzZsuWoX7cGco4jkcEWLjuxOUM6JZCe68BkMhAbamXD/izu+Wo1L43uxV//pPLtir18vWIfRoOBL27qT77DhdloICHCRkK4rXRMuT92p5vswqKA+8tbuSuDb1dKUOjuL1fTt2U0j5/XlXyHi/X7s4gKthIf7uek8vAmGDQJ2p8pmTpF+bD8fcksAMmaWfulnOR+dIGcfIIEMka+IBOedi2UjJtT7oQvrizuSVNs3VfSs+K6WTLuOGsvJPeVq/4bf5SvMx6THjwlz1lWlwvk5Dk/VQIhZU9kI5MDX+0HWVtUSwmmnHyzjI1O3QxfXC6BqnNflrXsWiTThT4dVbF0pE1xU9aSaVU9r/SWrJS3fZ6kTs+b4nsboxkufk+mD9mOsnmrqmBzcZCyTXwoIe6cwBPMDqzCM+w/OM54irzWw7EUpOLZOAND5k4JWJz/ipQ65R2WzJzeYyGhs2TxlAQorKES0Bj+tAT1krr5BkUjk6WcaPEb3m1BkVJuF4jRBHjkKu3wKdIzZ98KuVrbeqBkDgVHYTAY5fkydgZ+rMxdcr/yLMHSaPyTUdKv6qK35ThKWQ/DJ0sTYne5nlwGI/S/RSbUOfJk6tXgSfDz/RUf/+RbfDPzStgiJEMnvKlM8yp/mxb9JfDp92dyFDzuypsbu+z+L2crpRocu1NKq0xlxo83akHRkNDF20usrF5jwRqOpfi1ludye3C6PTx4dmcigyx8+beU4dvMRq7o14JbBrclLEgzcpSqkuBoKaX/4TbfYE7bobK9FkeQgwRyJkyYQFFRkU/AYdCgQdx66604HA6GDBmC2WzGarXyyiuvMH78eNatW8fkyZN9Hqtly5YYDAZmzJjB2WefTXBwMOHh4dxzzz3cdddduN1uTj31VLKysliwYAERERFcffXVNfp62rdvz4cffsivv/5K69at+eijj1i2bBmtW7c+8p2LDRgwgJkzZzJixAjMZjN33nknEyZM4J133uHyyy/nvvvuIyYmhq1bt/L555/z7rvvEhYWxnXXXce9995LbGwsCQkJPPTQQxiNR9+yWAM5xxmr2URydAjJ0d6Mmb0ZBRQWubnlkxUM7ZTAUxd1w2Iysi+zgLunr+bdq/v6ZNhEHSEdNthStZOa7IIi3v5ru8+25bsyuOztxTSNCuaNK3uxKSWXNRlW+pa9sh7XQaba/HSX98p8WIIEbxK6SGZJfEcpwdizxBvEAemX8/kVcpJ4wsXS+HXrbN8gTonCTFj4ipy4tThZTuIWvSond24nLHsfLvtQGh6XlHsYjJLd0/k8+PYmCRot8W3URX6aNDXO9p/VRHxnGHA77F0iJTIpa6XZ7VlPy2Sijy6QE7zmJ8uJYEk5zO5FkmXQawzEtJK+GwaDvMZ+N8lodX+6Xwbznq4Y6HE7JSgwfj6EVTIRSVVL23gJijmcHikpqoTLY+ClzFO4cccfhP820bvj0EbY8K1km4UmQEG69E364irvbayh0vz3x9tk5PbOP709aUqYbVKCV1ZhlgQmQuP8Z+Z0OV/KibL3S2+qmDZyTGbsgLVf4rnqWwx5aZLRZTDCrgWBX2BonHwAMVm8wciwRDlu/nxOvv/rObhhrjQ6PvsZCWCe/xrMesTb4C80TnpGFdkle8WRJ88b30lGpy9+Q35m0S2lZ09sO/j4Qt+1mKwyterPZyXw2eo06DMOVn8hwZ2eV0jJW3hS4NdTXUGR0ldrzxL/+7tdWusf0JRSNcNe5MZsMnp75PiZ/Nmo5B2GS6fJyePu4jHDRpP8zup7DRTlYYhtU+FuezPy+XH1AWas2U+ozcw9wztw3WmtcbjchFrNxIfbCKri50SlVLHIZnDJe3JcFmZLOVVofJ18RhgyZAgFBQV06tSptN8MSCAnJyendEw5wNSpU3nwwQd5+eWX6d27N8899xznnXde6X2aNWvG448/zqRJk7jmmmsYO3YsU6dOZfLkycTHxzNlyhS2b99OVFQUvXv35sEHH6zx13PTTTexcuVKLrvsMgwGA5dffjm33HKLz4jyqjj11FP56aefOPvsszGZTNx2220sWLCA+++/n2HDhmG322nZsiVnnXVWabDm2WefLS3BCg8P5+677yYrq5I2GUdg8HiOjct9O3fuZPLkycyZM4eUlBSaNm3KVVddxUMPPeTTqGjNmjVMmDCBZcuWER8fz2233cZ9991X5efJzs4mMjKSrKwsIiJqryaxLh3MKuTiNxeyN6PiSeVV/Vrw8DldfP7oVnb7U9rG8vpVvYkMPnLKbFqunSveWcLmgxWnGkQEmfn0hpO5+I2F9EwO5/U+KcTOvF6uTl/+mQQp/JVMXPUNeJDslRMukYyYss19ywqJgau+gzmPy4mpPxHNZJJOSKz0GmneT6ZKWYIlUwVknHlYEuCRE+PDm6V0pGlv6Sfyzyzfx0w+CbqPgpn3ln82aSx8w1z49kbvtK0SzfpIAOv94b7bgyKhx2hoMQCadJcZ8dl7JA06NF6+3E54LcDY50s/kpHTZacllXXhWxLMqsUa3KPRWI/F/ZkFnPPKfDLyHWy+vTXWt0/x7f9URsY18zmQb6DL10Ol7K28iKYSyFtW/Md980zJyjmtuGlxfrqU/zjyIDcVPEXevjEhsXDea4AHPr/c93HjO0rT4u8n+AZzkk+EEc9Iad+6ryssx3PROxhcRdIEvNM5Ejy89EOZsOXwMzVv0H1gCoIOwySQaIuQXgwz7vQGncw2mdb1/nA5FqNayfqCoyTjrDBHekUtfUeCMb2vluOn7M+o55XQaSSYbNJkOW2rvL7M3XBwA8R3kJ/bwldgx58S5B38gARtrGHFjdJrachj1j748FwJQJcVmQzX/AJRzWvneWtQYz0WlapJV767GJfbw8W9k7n3qzV8Ob4/J7aKqdM11OSx6Nm/CsP3t8Dpj8rvUUeefA7YvQRSVuM5dSKG0HhphFxsT3o+o95cREq279+rwR3jefaS7sSH6/AEpZSqKcdMRs6mTZtwu9289dZbtGvXjnXr1nHDDTeQl5fHc8/Jld3s7GyGDRvGGWecwZtvvsnatWu59tpriYqK4sYbbzzCMxy7EiOD+Oi6k7jhw7/Zesg7dnxk9yRuH9q+wpWTxMggPhh3Ile9t4SD2d6ygA6JYTxzSfcqBXFASr3O6JzoN5Dj9sj0LLvTzZKdWTwblcztl/1K4q4fMaVu9R/EAZj7H7jgLeljkbVPSiRCYqHPNTJ+3OORIMuydyUYk7kLLP6bWQHyASX5RCnbGj5FgiMhsZCdIlkqqz6B9K0SPIptA90vl7Ul9ZDbbp9X8TG7ni/TrHpfDSs/8p7Eh8TAqI8kjbl8EAdg399ychvXQZo6lyjMkoyc5R9IY9mdf8BJN4PTISe4302ANqdBu2GwdVbFx7UEBw7igEyqmPUI9B0nGQ7VaeSqKmgaFcxnN5zMddOWMWcPDD/hEgxrp1e4nafDcObtM3By2EH/QRyQrJiCTCm1W/yGTI06/WEpeyobgGl7umR0pW+XwERRvmTz/PqAZH816yPvrxKHN8Pvk2HM95LJkpsiDZBdRVCQBT2ulMyckkwajwdnaBPMe+ZLIGTA7dI/x+WAP/4Ll3xQHBQqzqAxGKD7aDkO3C4pT4zvKP1pMvdIyeDqz2VseEisd/KcySrNxPeX6ylUIqGLZNud9TSsmCbNvGPbQ7sz5Fj8/XEJFp1wiZQ3Ou3yOtZO920+HhovvXhKGPyXD9SIyGYSqFr1qYw797jkZ9Pn6kYRxFFKCXuRm2Cr6dgprTq4TgLef70gF7FsYfLZxGWHbpdhsOfIZ6qYVgA4nC6mLtxRIYgDMG/zYf45lKuBHKWUqkHHTCDnrLPO4qyzvKOY27Rpw+bNm3njjTdKAzmffPIJDoeD999/H6vVSteuXVm1ahUvvPDCcR3IAWgdF8ZnN5xMaq6dnMIi4sJsxIXZiAj2X0bVPjGc7245hd3p+ezJKKBtfCjJ0cHV+iNtNhm57KTmfLxkF1kFvj1eYkKtpV3GAT5flcaMjWbeHH0zp2x4lICnVSnrcHvcGFO3SFlCQhfofA789byUaICcoA15UK7Or/wEelwmJ4z+dBslJ5/th8nJcUE6tBooJ3/f3SzZDj2ulPKSvcvgrVPlxLTLBXDyeLhmJqz4GA6slJPW7pfJ5CuTVRrPdrsE8tIgNBbwyKjhWZWkEa77Sk5KywZySkS3hMim0HOsNFvOS5WMpKI8WPgq3DAHPE5pllyi9UApM4luLaUx/iSdIKOdV34oJTot+wden6qSjknhfH3zANJy7djbPoLNbMOw+rPi8fJG6HohhmH/Yag7iLC0tMofzFHc0Di+k/R1+XZ8xbHZ2+ZI8LLZidJvZsOPUlKYtg1+fVCyrvYslawze7b0WDrtbvjjGZlmFhQJuQcl8BHZAka9D6s+kuPG7YLwJDxD/g8ikuU9HhrrLVfcvwJ+ewwueF0CNbmHpBQqbbuMMC+50vvFGMgovo85SJogx98LG2dIZhzA/lVy3AQK5LQZLIHaXQtlHcExMrZ81efQ/RI48SZoM9A7ISu2nZRNRjaXZtCRxYGTxBOq/4/6b0Qmw6l3S3AXjwSvTP5/9yqlGia7001ksAVzcfZeo292HNVSLnb1uQaWviWff1oPlkl6C16CwQ/JsIhi6XkOvlsZoGQc+Hzpbk5uHYsxQF8dpZRS1XPMBHL8ycrKIibGm9a6aNEiBg4c6FNqNXz4cJ555hkyMjKIjj6+exHEh9uID7cd+YbFmkQF0yQqmH7/4jmbRwfz7S0DeG7WZn5dfxADcHa3JCae2RGj0YDFZKCo+MNQrt3Jmwv2MqBdh8CBnMhkCggmeMCdGOa/gGHQfcUjnMs0eE39R7ZdOxs6nQ2WIOl/serTcos7SU5ol74lTWVLpG2TK/ijpspJ8/pv5MTzulneBqwbvpOT3IvekYavMW0gfZvUmhdmysl6+2HQ8yoJ3uxcIGVRjryAZTaAPL4xwAneSTfLifKSd2D919Ks2BoGp94hJ82Ht0Cz3tIvx2AsbqgcL5lAw5+U3kHltTxFmtWWNM+deQ+M+c47tUcdtcSIIBIjgoBIGa992t0SRLGFS98bt5OIv6dCbNviUj4//XQik+W9PfsxaUx9YFXFIE6JlR9JcMNVBKfcLuU8IO+56WOg9SBpDmwJBpdTJsF1OEvex2X7TA26F3643bcBZk4Klh9uxnP+qxhWfyHvMZD3nzVEmnV/Mkq+73we9B4jAZmfJsLZ/4V3y5SOGc3QYbj0+Ek+UZp9O/IkaLpjnpRXrfqkYlZeUCR0vxQ2/Sy3/+0x774Rz0jZWXIfaWxeUlGcvl1KH897RRqZz3pYAignXAxDHq7bjBiTSca5K6UaJbvThbns1KrGPn48oin8PQ3WfendlrZNGuRf/aN8jgj0ecQPY21mNiql1HGolor+69/WrVt55ZVXuOmmm0q3paSk+DRpAkq/T0lJwR+73U52drbPl6o5BoOBNvFhPHtJD/66bwh/3jeEpy/qTqu4UOLDrDx6Thef2y/YlkpO23PkZM8Pz2l34y7MwvjtjRhaD5QGwOVHl4MERP6YIiern10OHUZIr5GeV8hJ3AWvy1SG/DTfIE4JRx4seLn4CjpyEvrPb3CS9/2Gxy0TiXb8Cb89KqUeJSfZCZ3lZHf7PMns2fSjZP6YbJIFFEiXCySjoGxzM6NJHqtJd/jrRWjeV6YODbwHhv9HMnPiO8tJ/p/PwaeXyf3jO0oQB+SE+cqvJIMJpPyk303QbzzM+Y/3uQ6ukwyQenBMH4u2UBn12qSHBP1sYXIl9Lf/k2lTQ/+v4n2MZpnotORNybza/JPveNjyigrky2SV93XZSWcej7wXZ9wlwcnt8+TKa2JXaTxcMv46OEreO/6mmACGec/I8eN2wWWfSPPggffC6E+lH47LLicF5mDpsWMNg00zvUGcpG5w5ZcSlFnxoQRW0v6RcqPLP5fMsV8fhIvfhY4j5ESiJCh6yQdgz4WQaN/sss7nyclGsz7w833+p0D98oBkuoH8XFZ/Dh+dH7gh+XHumD4WlTpKMn7cUNrsuKgOMnJq9VgszPQN4pRw5MHvT+AxmnymDsaEWrmgV7OAD3fZiS00G0cppWpQg8/ImTRpEs8880ylt9m4cSOdOnUq/X7fvn2cddZZjBo1ihtuuOFfPf+UKVN4/PHH/9VjqCMLtZkJtfm+HYOtZs7v2YwuTSN5fe5Wdqfn06N5FNm2poRd9hmmr6/xjtc2GHCfeCNZSQOI/nQE5KTA4Y1ywhrIvhXSxNfjlvKRrH3SC8Nkgd+fkKtRrQcHvv/Ov6QXT4mDa2ViVJPucvXf7ZKMG49LAjUlmTa2cDhzMnw5zjfT4bubof1w6XES267iFKmELpK9M/dJOPs5OZE3B0kgZ9MM+f+uF0gJWNlx560HQv9boVlf+X7g/VLSUtauhdI/ZPCDUu+evl1OtJe+45shZDSBoX6mTRxXx6LHIwEcgK2/SdPd0Z9K6VPGLin9Oflm6dU04lnIOSDBkcrGxIfGS0AmP13en+2HwVo/H9JBGgN/fQOYioNFw5+S91dIjLxXAsnaI5leboe8n8sGUTsMhwvfhqy9UoaY3FcCOfuWy/6gSHmu6WN9A1LfT5DGyZ3PgzMekzKr7P0w8D44daIc6/v+lrKvxW/AsP/gsedgMJqkN1DqVvj+Zrj4/cC9tRy5gEeO/5JAT9o2SFknvweUj+PqWFSqihxON5ayGTl1EMip1WNx65zA+7bPw+B2yu/q4mbHVrOJa05pxcy1BziQ5dsnZ2inBNonVPL3SSmlVLU1+EDO3Xffzbhx4yq9TZs23vGH+/fvZ8iQIQwYMIC3337b53ZJSUkcPHjQZ1vJ90lJ/kfKPvDAA0yc6B37m52dTfPm2oCyrkQEW+jTMpqXL+9FTmERGw5kc+Yri+nVLIyHzv+VZp4DWN0FBDXpjCE0noj0HXJiB9IENqySUoWwRO+J3c750HusXO0HKas65U44vCnw/Q0GpMtxsaa9pClg+jbod7M8xp4lUqpy2UcypSp7v5S3LHvPN4hT4p9foddVcuK8bzls+ql4pPm5klVTUpr19XUScLrwHfj9MTjjCalV/+7misGrHX/Kye+pE2HCUpmwFRzp3Z99AGY9JCfY31wvGRHfjvdf4tVxpDc7o44dV8eixyU9aUqs/BjWfwddzoO2QyD3MARFSQBj5t3e7JrzX63YDBsgvIm8BzN3S1ZVTBsJOmbtg93lAjPJJ8qH833L5Ht3kWTqFOVLkOa0uyXLpkkPCXqkbYXl78n7xxIsZXcfjKiYHbTlV4jrBHHt5H182j0SJEnsIqWIPUZLdpG/rKJNM+S4+e5myeq5+id5f6asg6BwCZ6u+waGPgLp2zBs/U2CRmHxEBIH4xdI2VqlDBWzdbb+JhO1lI/j6lhUqoocLjdmo6FOS6tq9Vg0VJK0X5INafYdbpEcHcJX4wcwc90Bfli1n2CrBHf6tIwmrhql+0oppY6swQdy4uPjiY+vWj+Offv2MWTIEPr06cMHH3xQOrO9RP/+/XnooYcoKirCYpG63tmzZ9OxY8eA/XFsNhs2m/7xqW+hNjO70vO5dqpcvV+0M5tzdmYTZjNjNQdz33Abo0+KgsNl+ohs+12mOK340P+D9h4Lf38g/394k2Q9JHaVviAhsVKGMfL5wItqO1RKt0B607Q7Q4ItG76H0BiYdo53GlRSd5n+ENVKTnbfHhj4cbfP9fal6XmVnDDPfgTmPuV7O5NNGjePmibZRSZL4AykdV/DqXdJOVV5RXlyEl5i1aey1pIx1SUik6XUyuUIvPZadFwdi0YzdD5fAnklHLnePk7dR0PeIZkMVdbs/5PeTQtehm2/SWAiqqVs+/Jq7/QnkPK5K7+SpserPpWrqj2ukBKv7yd4b2cwSmDxp4mSTZbUDVZ/Cn8+K/ub9JRSvsWvS9bXvpWBS7z+fl/KrUAaFjvypHHm/P9BiwGSARbIzj+lx9OOP+H9YXDTn7KW3IMSVE3sLH144tpLWeDuBXBgDSx7R7LjRk2VMoCy2WolgqL8T27TbBy/jqtjUakqKsnIqcvSqlo9FlufFnhfuzPwGM0Ygit+dm4WHcx1p7RmVO9kTCYD4UHauF0ppWpDgw/kVNW+ffsYPHgwLVu25LnnnuPw4cOl+0qyba644goef/xxrrvuOu6//37WrVvHSy+9xIsvvlhfy1ZV5PF4+HTJrgrbc+1OsMPbf25naOdE4sObSPmP2yWlFht/lGk0v/2ftxExyNV9o0nGLJeYMRHGzZAsmp/ulm3b5sKJ18vEn7KCo6W05avrJMAx4hkZy1yUD6fcAW+e4ntimLIGvr5eykNOurG4PMmFXyXr371Yvq6fI1kUZRkM8rpmPSwn5IMnyboDcTkCB2CMFu8YaYD134LZBldMl4yE/DQZPRqeJE2ix/3k7a2jak+rU6XBb9ngC0iPm8GTpMSuvPw0KWdq2hP6XiPvwfCmMma8/OPYs6Xx74Vvyvu5+cnw872+Y8iDIiXDbMcfcMUXEvD48Hzf7JYDq+Cra+CqrwGjNA8OxJ4j6wfJiMvZj8vpwHjZRxicdgkaBWr2bTT7HsP//CaZOuUnWOUelCbdySfK/pL7LHlLShK/usb3cQxGOZaWvuX7OAaDlHQppVQVFBX3yPGWVjXyZsfZ+6HvdZJxWVZwtATLPQE+wwBGo4GoUGvA/Uoppf69YyaQM3v2bLZu3crWrVtJTk722ecpTpePjIxk1qxZTJgwgT59+hAXF8ejjz563I8ebwycbg8p5Wquy0rLc+Bye6QPSL9bYNErsmPVJxI8Gf2ZXIk3GKTp6eHN8NU43wcpzJQAyx9lejIteVOatF78rpRu5KdJz5muF0p/kgtek/Hhv0+Gs6ZAfAfpV3LphzIda+HL0sS4xOaZ0q+m87mSJeNP60FSSlPC7ZRJUZtnwv6VkmHR5XzYNk9GNG/8QV7L6Y8G/gEGR4Ml1P++sHjodhms+liaMCd0kTVPv1oaKw+6H769SV6PwVBcUqZqXWQzuHoGzHtaGk66iqDlqfI+s0VI1lZ5wdHgLPBmy4D01tm92P9z5B2WrJ3wJGlkfGC1736PWwIouxdLP6m9y/2XKDntsOozWVeT7oFfU0RTKXkEPN1GcbgApq/2MKxTa9rHujD0HgexbSQryFUkQZ/NM2XiVqvTYPn73seqLDNs9WdyjIx8Hr65UTKFTh4v48uv/EqO5dTNkkF30g2SRVf2Z2QwwkXvSk8qpZSqAofLjdlkxGAwYDRAkbuRjx83muTzwcXvyueV/HRo0R9anQILX4VzXqjvFSql1HHtmAnkjBs37oi9dAC6d+/OX3/9VfsLUjXKYjIyrEsSv230UxYBnNwmlrAgE9iCJCMmti38+V+5opSyRjIBOp8rJ6sgfTOGT4G/npOpTqHx0vfD7axYevHHfyUDpeNIOPE6Ke14Y4B3v8EA578u063WfO692p/UHS58S8pUcg7INrdTvgbeC7sWePv5lOhxuZxMl4yajm0LKasly6HN6RBfIOv75gYpFfnqWm+fn7xDUhp2cH3FH9Cpd8qHMr8/3BBpsHzCRTKNaO8ymQ500k2y79eHpKTmgtchNFHWlpcKoXH+H0/VnOiWcM7zMORBCbgEhUuwxlUkGTR7l/nePigKcnz7gPktGfLZ75JJU448CXqu/8a7z54j/XXOfVmah+/4I/Dj7F4El0yV7K7o1r6To0r0nyBBynNexJVwAs/8coCvV+zjk9VZ/HpDJ8JPvBbD19d7p2IZzTJJ7sqvpY+V0+59rJYDYH6AE4mCDEjfAfOmyNoNBplO53ZKcKjjCMnYSegqWU8n3SiT63b+CcExUlIQllTaxFMppSrjdLlxe8BikgsdZqOx8WfkxHWAn++XiwOdzpHf6/v+hgX/g0ve92ZXKqWUqhfHTCBHHftOaRdLYoSNg9l2n+0Wk4GJZ7YnzFZchx0WD33GQceziq/q2yC8XNPjsHgJVHQ5X0Yfm4PkRDnvcHEJR7mT3/x0yQpof6aUHpXV+TzJlFn1ie/2lDXSV+T0h+D7W2VbTBt5jp/ugnNfkmat234HWySccKEEnn4vnkBhskiWzZzJ0rMnN8Vb4tXzCjkhLjuF5/cn4KJ3pERka3F/FFs49L8NTEH4NGYuz1kA39/iG1ha9i6c9YwEwKxh8vgl+xNPkCBVQhco14tK1TBLCESVCyiYLNB3nPR/cZbJVMtNkWyWsjwe+fcrmfBWXmzxBDNrKAx9FHbN9wYzDQb5sP7FVfJBvrLm4aHx0hh5/guSCbPgJW/gJyhKysGa9ZGAaFgCZmso158WzIYDWbw3IpyItFUw8x7fkd9up/S4Ckv0LS/sNUYCUIGyclqdBjvmyX0imsEPE7zHtMshGTglmp8kfXXCEiC5T+DXp5RSATiKgzbm4r+HJqOhTqZW1Sp7Lpz3ilzwKuk1GBIjUzcLszE08penlFKNnQZyVKPRLDqE6Tf15+mfNzFrw0Fcbg89kiN54vwTaB1XrmzIYJBMgsoYjd5mpm6XZN4YjdDlQillKc8WLid85XuNdL0QvrvF/3Nk7CweCx0u2Q3DnpSg0eAHpfFxZHNocbL02UnbLiUfCV2lietJN0nWTtpWGQc+eJJM/sk7JD1Afpnk+1wFGTLSvPdYGHCH1K8XZEqZiTUMeo/xv8bCbPh5UsXsIIBZD8KNf8Cbp/puP7hOJhONny9ZI6ruRbaUfkXf3eydUhWWJBkmST0kkwukMXH/W+T9Xd4Jl0gApkRMa7j+d2myvOknKeXaPk/eu5tnwnmv+gZByhpwu2SDbfhO7tNnnPRRcDsloBrdSkaOl9EmLpRvLm9O8GcXSQlj2SBOWUvfhgvekCDMSTdKYDM4Wh6zfImZyQp9r5VsNZAg5eHN0ui7zzVyzDsLpVTgn9kyHS6ufaCfslJKHZHDKYEci0kCOWaTgaI6mFpVmwzrv4G1X8rwg+FPgdMBeKTkfMN3cOvfR3oIpZRStUgDOapRaRkbyrOXdOeh/CJcHg/hQRZiaqKhXlG+lFXsWyENj9P+kSauJaxh0vy3MFcaJa/5wrvPYJT7B5K1R8pgTr4FYlpJA1ZrBHx2mZyEBkdD76uhw3AY9pRkNRiNkHNY+vHc+KcEb6wRcPaz0sfHFiaZQ+U5cmV6UGGWnJSv+lR66pz/mmRc+FOQDlsDNKh1u6SkJaaNBJPKsmfLyX7/AEEsVbvMFgmMjPtJMsbwSBDjm5vg3P9J4OafXyX41/JUacg9/0UJ2NkiJMhy4g3ecsMSUS1kOlmvMRKE+eJK2V6YBXsWw5CHYN5TvqO6T7oRYtp6e/MUZkr6fVntzpDeUWXehzaLCXbNlRImf6VYJQqzJKun9SBo2ksCOY5cGPO9POfaL6XUqtso6fO0+nNvLx+TVfrgJPWQNaVtldff60q45AMqzVRTSqkqsBcHcszFpVXHREaOLRzOf1UuMP14hwTAm/aCUycWl5A38tenlFKNnAZyVK05lF1IdqETs8lAdIiVyGDvCMrcwiLS8xw4XG5CbWaSIoIwVLGJbliQhbCaHmdpskFkC9i1ED66SEpDQuPh0EZpBJvQRcaR958gdeNXfinNf62hUjduCQkczEnqDm3PgCbdihu9GmDGHd79BRlygpm1WzJ1CtLlsTbOkKlbY76Vk+a//ivriWoBgybJqOjFr/l/zhMulmyDi9+Tx/v1IZlE1WuMBHjK9rdxO31PyssrzJL7+rNrvpzEm/RXyVHLPiDlds5CKe8JTaheb5awBPkCSYXvOVr6Mg2fAr2vkg/cbpcEKYc8JJOo4jpAbDsp0fLHYJBgodMhx0WJRa9Bzyvhut+kj01+ugROts+FzTNklH0gjjzJjInrII8N8r7b8acEl6JbBb6vLQJCoiV7LecA/PW8BHZOuEjef6fcLiWP85+XwGrn82D0J/DDbZKZl3gC/HCr9/Hs2bD4DTi0yTsOXSmljlKFjByjofH3yOlyAXw51ne65/6V8OXV8hnIFFy6+XCOnezCIkwGA9EhFiJDtH+OUkrVNj37UjUu3+Hk710ZPPzdOnalSXDj1HZxPHF+V9rEh7EvI58nZ27k1/VSHpUQbmPSiE7EhFrZdjiX09rHkxgR5BP4OVqHsgtJy3NQWOQiNsxGXJiVEKuft73ZKuPE1xRfyf/mBsnCiUyWk9ohD0u/j4PrZIRx2jbpgWMJgSY9pQnywlcqPm5UC8mcKZl+Y8+Rk1B/1n0jzY4/vUyes9soGPudlKt8N94bbMncDbvPlxKYzTMrZjL0GC0nzcl94dcHpQltiRXTJJgz9P+kTxBIf574TnB4k/91NesTuKlsbAcN4hwtt1veQ19cCVl7ZZvJAqfcJdkyR9NM2hYG3S6F5JMge58EA1d/5nubIQ9Bq4GBgzhlma0y7Wn1p95te5fBidfD4S3wzy+SqeV2Qfth0PZ0CXD60+4MWPUxzt7Xst6VjNVkxO320D6mI9YN30nmTHiS/xK/k8fD4reh01nw8TjpbdV7jARnBt4nDTkzd3lv/9fzEjC6+icJ7JSdRFfW9rkS/FJKqX+hJCPHYvRm5DT6qVWZO32DOCU8bvjjWbjkfQqKXKzek8lD365l22EJ5PdrHc2TF3ajbXxYlS/QKaWUqj7tUqpq3JaUHMa+v7Q0iAMwf2sql721mJ2peYx9fxkz16bIuHDgUI6didNXsy+jgK//3sewF//ktTlbycirZLzwEXg8HjanZHPpW4sY8dJfXPj6QoY+P49Xft9KWq7d/51iWksmTsl0J0eujCjufrkEYEDGj391Dfw9FdoPlyv/a6ZLI9ieV8pJY4nErnDFl9C0h/ekvDDLt0FxeZm7pdTK7ZLykPx0mP1IxYwZtws+v0pGKQ/7D7QZLOOhL3xLeuwU5sgJcdkgTomVH/kGbcLi5XUb/Pw66Hpx8UhrPz8zgxF6XRH4tajKZe2Baed4gzggvWT+/K9MKTtaBiNYw4sbXd8qfYyGPyUByFuXS9lUSHTVHy+6tWStlBwXA++VrJilb0nQpmRK29bf5JjwF4CKaiGTojBg2vQDrdz7mDpvPY//uB5X14vlsef8R0oA4zp472c0Sb+bqBYQkSRlVEUFkhn35TUSSN2z1DeIUyJ1i5QF5qcF7r0DFUeuK6VUNTlKS6vKNjtu3Bk5nq2/Bd65ZzEel53th3O58t0lpUEcgCU7MrjkzUXsyyyog1UqpdTxSy+lqxqVme9gys+b/FbqHM61s2h7Gmaj/ys0b/65jZsGtuXh79bx9l/bGdwpngFtj27E9f7MAka/vZiM/KLSbUUuD2/8sY2kyCDGnNwSY/l1BEVKRkzboXJy5y6SevDQBMje63vbXldJNsza4qbIS9+UE85xP8nJpzVcSrNKsl5KBCpRKruGslOICrMqjkMvkb0X0rdJD5S49hJsmfWQjAY//RHv2vxZ8qZM6zHbip/jMIz+VLbvXSav+cTrJLXaEgyn3iVTiDzFH0wtIVK2Fdm88tejAtu1wBsgLG/uU5LdEp5UvccsyJRpZrMf8f5bmaww/GnodgkER1Z/nUER0H00tBki2WFmKzjyK06M8rhl6tRF78C6b2HjD9Lrqdtl0Ols+PgiyEnBAET++V8mnfE/ntndkZeWF3DHxZ8Q/P110nR7yIMSPPS4JYCzbS78cDuMmibjxOM6SpAyfTt0vxQ2/BB47eu/lsbhBqP351FeaGz1fyZKKVVGydQqS5keOUWNvUdOUCUB/6BI8Hh4YfaW0otyZWXmF/H7xkNcPaBV7a1PKaWOcxrIUTWqwOFi9d7MgPsXbE2lXUIYmw9WPIHdk17g07j47T+30yM5ilBb9d+ma/dl+QRxynp1zlaGd00kKTK44k5rqGTmlB/h7E6CTufBph9knHFwlG+gxO2Cpe/I1xXToXk//wsLiYMWA2D3wor7gqOlkbGjTJ+RI6Ulm6ywdyns/Mt3u9nm+zjlOXKLxzHbIH0HfDVOxor2vFJOjguzYM2XEBIvPVdOvVumYR3eLEGcmNYSZDBpHfxR278q8L6sPZKdU11p/0gwryyXA2ZOlDHjbQdX/zFBevaUHBdrv5JJUP7KoNK3S2ng2B+gZX85TrL2wUcXeDN3ADweon+7k/Gj/+D0qXsZ1vVEWo6ZR5QrFeOmHzCAZINZw6U5uMftzfSJaCqljSCPWVmJmNEiJWbtz5SAZ3mWEMmcU0qpf6F8Ro7ZaKSokWfk0O4MyRD1p8doMAexcndmwLv/seUwV5zUAotZk/+VUqo26G9XVaNMJgPx4YGzTpKjQ8gs8F8yFWI1+VzZSc9zHPUHoQ37swPuO5xrx1HdK2UhMTDyOTj9Yelds6aSbJeFr8hIb7+PEy1TICKa+W63BMO5L1ec9JOXGrgJrDVUvvyVPe1dLj1LAul+mdzX5YJl78m2/HRZ+3e3wC8PwP4VsPB/0og3KFwmV3UcAW0GSaaEBnH+nWa9A++LblW1HjZlFRXAAj99mkrMfyHw+7I64trDig9l3Lg/CV3g0HoZi27Phd8e8w3ilPB4iNrzG+f3aEbzcAjd+AWmqSMwLH5DGhG/d4YEpUqOOXuONDtOWSONmkEaenc5P/Bau5wnx9RJN8r7tyyTFUZ/BmFNjuanoJRSpRzleuQYDeCubIhAY2C2Sul2+QtKzfpItq7HRXxY4M97zWNCMAXIwFZKKfXvaSBH1aiE8CBuGdzO7z6DAS7o2TTgFZwLezVj5toDpd8P6hBP2FFk4wB0aRoRcF98mA2r6Sg+XIQnwikTZWRzYWbg2xVkVJ5NEdsWrv1FRjH3uxnO+R9c9jEsfLliv47MXdI3pHxJlsEA570i2RH+bJohpV7BflKjY9rIWHOQ8rGcSvqH5KeByxl4vzp6LQZIero/Qx46cllVXqr0finIlO+d9oolgGXl7Pcf9Kuu8KbgMch7Y/iTEJYo200WPN1GSWnUb4/LNrNNsr8CiHCmMbnrXuIWPEaQzQZXfOEb4Fr3FeSmQJ9rZPztaXdLhpE1VDJzDqySHj5Ne1V88JYDJECasha+HQ+D7ofLP5NG0mc8Jplz1hBw1cDPRCl1XHO4JFjt2yOnkQdy0ndIwPuK6fL78+Sb4ZL35bOF0wFuFxOG+P+8B3D5Sc0rlrArpZSqMRrIUTXuzM4JXNTLN+PEYjLwyuheNIsK5v1xJxJiNfns79c6hoEd4pm14SAAEUFmLumTXPqhqLq6NYskOsR/RsNtQ9uREB50VI+LySR9RkoCIf60PxM4wge4zN3wyyRIWQ1rpwOGiqVQnUZC14tkfPK1s6RxbatToffVcM2vEgjocZlMxSqv5QAJCl32sYwpt4VLVtGA22HsjzKNC8ASJE2SA2l5qox+VjUvqrn0VCqbKWIOgqGPQruhge+Xlwrrv4Np58LrJ8vUq91LAKNMowqkxSmSWfVvhcXDqA8gYxds+BGGPgKXfoRn3M/YBz0MM+6UyW8AhzbK1dsAjE26E/bLnRhWfQJzn4SvroXT7oH4jt4bLX5DTiZyDkhD5FHTYPsfkk0T2xZ+vF0aOJ/9rPQV6nwuXPi2vO9n3iuPkXdY/t+DBHZWfy7lXu+dKRk+Sin1L3jHj5dk5BhwNfaMHKNZMiELs8DjAks4ZO2XzxLL3gGDkQHtYrmyXwufu5mMBp69pDvJ0SH1tHCllDo+GDyexv6Xpm5lZ2cTGRlJVlYWERF6ghtIZr6DQzl2Vu/JJNRm5oRmkSSE2wiymHC63KRkF7I5JYfUXDsdE8PZmZbP/V+vweFyc1r7eB4Z2Zl2CUc/utLj8bDlYC7jP/6bHakSILGYDNxwWhuuPbU1cZWkAx9Rfjoc2gBfXFVxAlVwtDQBzk+TIIy/0dy5h+HTS6V0qURYgjQUjm4NJhsYkBPl3ENycrplJvS5VrIRzMEQEiX9PYoKIW0L/P6ETOgJjoGTb5GSEoNRMhIsIVCQLv133C7pGZK5C6JaQVSyjMF+Z3DFpsomK9z0FyR0OvqfVS06Zo7FnBQJzrjs0kMpLFECbP4U5sD852H+ixX3jf4UmvSE106sGBQ022R6VdmJUP+WI0/ey858sIZjD0kgNc9FjGM/tsUvYdzwHUQ2g2FPwScXV2w2nNQNTrpJRoiXFdEMBj/g3Z54Apz3MrxzunxvMErvhhb9IflEKUFzOSCqNXiKpI9T5m4JlB1cL5OunHYYPAn2/S3lWGU17S3T37Tp8VE7Zo5FpY7S96v2ccfnq/hg3IkEWUz856cNtEsI46XRfrIFa1FNHoue1H8wzJ0C3UdBUb6Uyka3gm2/Q7szIaYtRDYlq6CIwzl2Vu3JIMhionuzKOLDrQRbtQ2nUkrVJv0tq2pFVIiVqBArHRIrZgCYTUaSo0N8rta0jg9j9sRBMtEm2EJEcDX7g5RjMBjomBTOFzedTFquA3uRi5gwK/Fhtn//4SIoEg5tgqt/lCk6W36R8eAdhkvZ1a8PQu5BaXgc3bLi/R05vkEckCDKLw/I//e9Xko+Fr4svUBi2sjkn8jkirXqliBI6g4Xvy9XzowmmThlLJPJ5PGAKUiyJH6ZJOstEdMGrvoGrp0t/Ug2z5QT7uQTJcMhtlxfEVXzwpOqPp0q77BMD/Pnp4lw/Ry45hfJUtm/UrYnniDle4F6LR0tayjEhJZ+awOaeXIhP1cCRue+JBlGkc3h2l/l/b1vOVhD8fQai6HtYPj6hoqPm73Pdzx6p5EQliRXh91OeX/+M8s7oj2+I1w9E/IOwYbvZerb1t/g4DoJgo6aJg2XC9IqBnFAjkWnjslVSh09b7Njb0aO0880p8bEsGuhZP3OegTanyEXkdZ/K+W1RlPphM3IYAuRwRbaJYTV84qVUur4ooEc1SCUfBCoaQnhQUdfRhWI0SSNhPf/LWVPF78v23f+BV9e7R0pfXgTbJ8nH4B8mhsbJHOnfDZPibB46cFjskofnISuRx4bHRQhX+Vl7ZV+Oas/l8yFLhdA1wtgxkS5wpa+XTKLxnwHF74pgQKXUwJJEc2OPDVL1a3UzYHHaOekyPumSXe44kv5tyxIlz4HuxaDNUz+XYOjAvfm+TdcTrlSO31sxX2XfACXfy4BE4MJgz0H3jyleHJagMcC7yS14Gg462kZb16W2QYXvCnB0c0zYessydjpfilE3CVlXvGdJcj64d3+nys4Wu6jlFJHyeFyYwBMxX8zjUZwNfYeOYld5YLAyTdLvzJ7jpSVtx4EBzdAZIsjP4ZSSqlao4EcpY5GdAtpLLvsXfnyx10kmREJXaR0I7I4mBMSA33G+S+PMRihw1mSWXDzQsnoOdrpUFl7pY9K+nbvtj1LJVPo7P/C98WlKwfXSTaDsxBm/5+MMw9NgAG3SeAnPPHonl/VPMsReg6U9EvK3AVTz5ZsrAteB0cWfHKRlPy1OAXOfAziOgYu4ToauSnSkNifH26FWxbLtDOQ0ieTxX8gx2yTr26XwpAHvFlt3S+VpsYLXpL7N+8HJ14nj/PZZVJSVWLvcikzG/mCrKnvNXJs+QuCnTwBQvU9rpQ6eg6nG4vJWFoObjoGMnKw58L2ubDpJ++2A6th9WdSQu5vGqFSSqk6o5chVYOTmmtn3b4spi/bw7zNh9iXkY/zKMeQ16qo5oEzG2LbSjkHSD+dzTPlpBrkPr3GyFWtsoxmOPd/0idl8AMy5vlogzguF6z6zDeIU2LPEsl4iCpT9lWQCe+eKVlFTjtk7YGf74Of74W8tKNbg6p5Ma0DB3OSukFIrJTpfT9B/h2HPgorP5Y+MdnFU6u2z4F3hkjT35qUnxY4y8yRJ+WGJUITYPCD/m878D4Jwpz7P99G0EGRkNwXLnoLxnwLA26Fv56XyW1lgzglDqySrKSo5vL9qKkVG4O3HQq9x0gTc6WUOkoSyPFmsBoNBlzuBvi5pTpcdt8gTom8VFjxoW8Jt1JKqTqnGTmqQUnJKuD2z1axdGd66bZwm5lp155E9+TIo55iVSvCmsAlU+HTUb6ZBdZQOO81cOTDiP/C0rdhxTToeiGExsltMnfBsMmQc1AyYIKioPlJcoXL4wbzUQZwShSkwupPA+/fNAPaDJZ1GQzF2Qp+rq5t+B4G3quNYBuKsCS5EvrFlb7ZJUGRcP7r8v46vFnK+qyhEqzbPq/i47hdUqY05hsJ/tSIysvw3Bi9Vw4sQdDrKsnQmfsfSNsqQZshD0uAs7L3myVEAo8fnA09r5D3aCCbZkDn8+Vn0X4Y3Pq3BDIL0qVZcmSy95hUSqmjVJKRU8JkbPwZOZ5/ZgX+rb7pJxjyUF0uRymlVDkayFENRmGRi1fmbPUJ4gDk2J2MeW8Jv941sGGNszSZZcz3LUtgzeeSedO0j2QNzHoQ9q2QZsWnPwLb5njvl3MQZtwlzYf7XAMtTwFHLvz5X9j6u/TFib6q7l5H14t8GyCXt3uxZHuo+me2QtshUna38hPI2C59DJr1lSukA26T6WQA8Z2kuXAgB1ZBYXbNBXJCiydulc28KREUhT0ojuCy20JipF9TywHgKpLjKayKJU7p2yVrrCraDJZeVgYDxLSSL6WUqkF2pxuL2RvIMRoNOBt7j5wjBOc9BuMRbqGUUqo2NaD0BnW8S82x89Xfe/3uy3O42LA/u45XVAWWIIhrB6c/DMP+AzvmwYfnSRAHJNPgm+uh91jvCbPLARk7ZYz5X8/DxxdJg9h/ZkuWxf5VMlY8Y5eMFN+5QHqCOO1VX1dIHPS4IvD+TiNh1wLofbWU3yx+PfBtbRUnj6l6ZA6CA2vg0HrJNNk2R8bZL3tH3ksmi0yNctplfyBGszTuriHpxhjShr9asXzJYCT1zJf4eadHUvIPbZQ1p6yVMrCwBOkfVdUgDkDaNvnvtjnQ+ZzAt+txuQS6tGm3UqoWOVxuLEbv7xmTwYDL08gDOV3OC7yv64V4yv+uV0opVaf0t7BqMBwuN3Zn4JryfZkNeESwqwiWvQc7/vS/b/l7kiFhC5O+N9GtJJjjT+dzYc0X0qOmeLwnlhAZ59zxbHmMIzGaoOflUl5Vvk9OiwFSxtW8n5xEm4OkV8iWn/0/TvN+R34+VXdyDsDvj0nPm/LStkqmyoVvwvsjoGlvCWL4O6HocmHgbBx7rgRkyjZDLswu7oHjkVLA4Cjfuzg9TF4RxgOjfydu08cEp66lILoTaV3G8srKIm7vXwRfjIHdC713SugCoz+V3j/VEdtO/rt3GZxyh4wgL98np2kvaHtGhXUqpVRNczjdPqXfRgO4GnlpFZYQ+Tux4Vvf7aFx0H8CbrdHrwYrpVQ90kCOajBCrCYSI2wczPafedItuRZGJtcURx7sWRx4/95l8M8saYIc2056gXxzfcXbRTWXQM+Pt/tuL8qHb26Am/6EJj2qtqbIZLj6R9g0U6ZMmCxw0g3SGyQkFixlCl3OmgIpq32DAwYDnP9G9TIlVO0rKvAfxCmx72/odwuM/wt2zIcznoDZj/jeJqoFDH2kYsZO5h7Y8its+E7G2fe7GRI6S0+ZXx6Q8eIeD7QeDCOelsyf4qyeYKuJtEI4fdp+hncaRZv40ezKdvHLR4e4bUACTeY/5BvEASlH/OJKGPOdBBWrKqa1vIbM3fDDbXDuy5C6BTb/JP2e+lwLbU+HiCZVf0yllDpK9nLNjk1GQ8Mc0lAd1lBofRq0GVRm/PggGUF+eDOeVoPre4VKKXVc00COajASI4K4/6xOTJy+usK+zk3CaRHTgPrjlGe2QWQLGXvsT0QzOWlN3QLtzoD4DnDtLJj3lLcZbVRLGVP+26OBn2fhq9JDp6pjoyOTJXjT7RI5wQ2UnRDTGq6bDbsWyol8dCsZ9xzRDKwN+Od+PDJZ5QO2I8///uhWYLZIlkp0S8jPgHZDYc10yN4nZXXJJ8p7o6yM4pHlWWXKGzf9JH2cIpvD1t+823fMg3fPkGBR8WSpqBAr95/VmYvfXMiM9ak+D31OOwvGT3/1v96D670lVlUV0VSCP9PHwsF1chwNfkgaGluCIThGG3QrpeqM3ekql5HT+Jsds+UXWPyGDDw443EJ4tuzZQpiTgqWsScC2ixeKaXqiwZyVKmCIhcpWYXM2XSQvRkFDGwfT+cm4SRFBh/5zjXAYDBwescEnr2kO8/8sonUXAcmo4GzT0jigbM7kxBexeBFfbAEyzjk9d/433/SDWAJg7mTYd4U2RYSI9kSw58GjxNC4yW7wd/I8BJpWyUjo6qBHJDMmpCYI98uMlmCN90vrfpjq7oXlgB9r4WFr1TcZw2TkqIS5iAOGaLZkWtlW/i1RCdZ6ZIUTozNhk/no6JCWPA/3yBOib8/gMs+ljT7onzvdkcuLH0XznisdMpap6Rwpl5zEo98t47d6XLbU9rF0izYWfFxyypIr3y/P7FtJZhTmAVZu+HHO2UaXMm+C96EJr0kqKWUUrXI4SzXI8doaPSlVYbCLLjgdZjzpEw5BCmrGnQ/HNpUv4tTSimlgRwlCotc/LH5MLd88jclnz0+WLCTVrEhfHx9vzqbFhUVauXi3smc2j6OPLsTq9lEbKiVUFsjeKvGtoeRz8PP93vHkRuM0sMjNF76g5Sd6JOfDj/cCpd/AR3Pkm1FBZDUo2K/jxJNe1fewFYd+8w26H8rHN4C/5TJcgmKkoyu8Kalm1KyChn/8d+s2pPp8xCPndeVi3s3IzyoOMhRkA6rPw/8nNvmSknett/Lbf8dTpsIZrkqG2IzM6hDPF+N7092YRFmo5HoUCuWgr0SpHT7GXEPR1++FxYP+anwySjvMQfSDHnaOTB+oTQjV0qpWlShR84xEMjxdDkfw8cXQ36ad2NeKsy8Fy55Xz7fKKWUqjeN4OxY1YVDOXZu/XQF5T937EzL59lfNzPlwm6E1FEwxWg00KQGs4AO59hxuT2EB5lrNyAUFCGTopr3kxKq/DQpc9nxp0yi8jeWGaR/SbPekmlhCYZT75R6dE+5+nqTBfrdWJr9oI5j4UlwwRvynjq0Ua6SxrSRkqPinjVOl5tPl+yqEMQBeOyH9QxoG+sN5Hg84C4K/HzuIjD6+dAeEiulXuUkRASREFEma8yUIMfGyo8qPkbrQbL+o+EogPn/8w3ilHDapcn4mU/g8JhIz3NgNEBsqBWTSU9AlFI1p0KPnGOhtOrAGt8gTlkLXsJz6Uc6flwppeqRBnIUAH/vTA/4oeOnNQe4Z1hHsgqLWLcvm3mbD9EsKpizTkgiKTKIEGvDeBt5PB72ZRaweHs6a/dkcnGfZFbtzeSDBTvJyHfQv00sd57RgdZxIVjNNTd22Yc1BGwR4CySniR7l8lEqMJKRqenbvEdLR7TRjIrvp8gE4pAyp4ueFMCQ0fitEvPEZdDymG04euxKTRWvhK7+N2dmutg2qJdAe/+/ap9NIsK5p9DuVx7YgLJnc7FEKg0sM1g+OnuittPuVMCmH6UPR7X7Mlgwon3EI8B4+pPJfBiMEDHc2DYf6AgS8q7wpLAVMVjMydFjqv9fwe+zZ4lZGVl8NyfB5m5NgWLycjok5pzad/mNI2qm5JRpdSxz14uI8dkNOB0N/Jmx7srGeCQshY3UEufpJRSSlVBwzgDr2F2u51+/fqxevVqVq5cSc+ePUv3rVmzhgkTJrBs2TLi4+O57bbbuO++++pvsQ1ERn7gq/FOt4d8h4sbP1rOrjRvj4xnZ23mpct6cmaXRIIbQDBnc0oOl769iOwCJ5NGdOKF2VuYt+Vw6f6f16Xw+8ZDfHPLAE5oVosTsMKbSPPgfuNh/wrIOwwtTg58+7AEMJX7+dki5ATXZJUTXkcB2MKh7PWv/DRwu6WBsak4syL7gPQ6WTFNyrQik+HMJ2SCT3B0Db9Q1ZB5PB5yCgMf1ylZhWxKyeH3jYf4Ytke/rzuHuK2/S49Z8pqP0zKtgoyfLf3vU4yyQLYdjiXuau3Em52k5PvZPCbe7m67zjuuOkOglx5GCzB0tPmg7MkYBkcLYGhnldU3vjYaZem4t+Nh44jpBFz6j/+fwbRrXnlz318tNg75et/v/3Dj6sP8PH1J9Vo5p9S6vjlcLoIK5PxazQcA6VVcR0CZ9xENqfIY9FAjlJK1aNq55d/8MEHfPnllxW2f/nll0ybNq1GFvVv3XfffTRt2rTC9uzsbIYNG0bLli35+++/efbZZ3nsscd4++2362GVDcuJrQKf5LeND2VXWp5PEAekGuOu6as5lON/XHhdOpxjZ8KnK8kucBJiNdEyNsQniFPC4XLz2I/rycx31N5izFZpONv8JLimeAJUs77S28SfUydCaJkeIek74IPh8PV1MH0MfHEVfHsDvHcGZOyUTIQ10+Hji2XK0NynZOJQXip8cyMseVOCOCDNa7+6ViZReRr3h0pVPaE2MwPaBi5ZOrF1DBv2S6ZYvsPFVd8cJvfq3yUAGd0KkrpJFth5r0rA5pYlMOxJCQzevEjGlwcoiSrIPEj0njncsPt+Rq+7kaeCP2PW1cks25vPoHd3kh3aClZ8CJ9c4s06K8iA3/4P/nwe7AEmcoG81z86X0aPr/sael4Z8KbZvW7isxUVSxq3Hc5lyfajaLCslFJ+SGlV2YwcaOzTxw0dR3gvEpXjOeUOCoN0MqBSStWnagdypkyZQlxcxQ/vCQkJPPXUUzWyqH/j559/ZtasWTz33HMV9n3yySc4HA7ef/99unbtyujRo7n99tt54YUX6mGlDUvTqGBOa+f/pOyRc7rw/Cz/zXddbg9Ld9T/CVF6noNth3MBmZyzYldmwNsu35lBTqG3p0au3cmutDw2HshmT3o+dqefhqwej/Qjyd4P9pwjL8hshagW0PxEORlO7ApXfg1B5TKBeo2FEy729h9xFMCCl/w3hXUVwdK34O+P4JsbYP9KKcua/4JkNWTsgp1/ym2N5TJ8Zj/qPWFWDUdRgbynclLAFaAR8FGKCLbwwNmdMBsrXlNtnxCGxWTkQFZh6bZNB/O44uuD5Ax8FK6bBWO+h56XQ3iiZMskdJLJbKfcIeVcgTK88jOwzX+G2B/Hwp4lkLaV4NUf0PyLYbwxPIRJIzphs6fCkjf833/5O5B3yP8+p0MCla7iTKO8VEhZI1NUyr7nTVbc577Ch1vM5Dn8/1y/XrGXfMcRpmkppVQVOMoFciQjp5FHcrL2wvmvF2cDI/3XDAboeSWG4BiCCyteLFNKKVV3ql0Ps3v3blq3bl1he8uWLdm9e3eNLOpoHTx4kBtuuIHvvvuOkJCKU5YWLVrEwIEDsVq9zTmHDx/OM888Q0ZGBtHRFU9M7HY7drs34yQ7u5JeJ41YbJiN5y7twWdLd/P+gh1kFzg5oVkEj4zsQlJkEJsP5ga8b2YlZVl1xVHmJLjI5cFmCRyjNBsNlJzbHsgsYPKMDfyyPgW3B2xmI+MGtOL609oQH16cQZNzEDb+CItelXKmVqfC6Q9DTDsoSPM2AwyJlbIqQ7kTZ1uofIXEwvgFUk5SmA1x7WWaVXCU97ZFuXBgZeAXum8FtB7o50UFSRnX6Y9A0gkSbLKFQ8o6CQzlHqxaAKoBO6aORbcbMnbA/Bdh0wz59+szDnqNgchmNfY07RPC+G7CKfznpw0s3p5OsMXEhb2acUaXRO76YlWF22fmF1HoNhMefpRTpAByUzAuf6/i9qJ84v98BEfbKaQeyiHZX4NikCBmfqoEQnNS5P8NZgiJkWbg+1fI7bqNkiBoUb6Uft04T7J0TFaI60COKZrv5q0IuMwQqwmzvwbO6oiOqWNRqRogPXLqfvx4rR6La7+Uzy3X/QaFGdLHLLyJZAZ/cRWWCUtr7rmUUkpVW7UDOQkJCaxZs4ZWrVr5bF+9ejWxsfWXZunxeBg3bhzjx4+nb9++7Ny5s8JtUlJSKgShEhMTS/f5C+RMmTKFxx9/vFbW3NAkRgRx65B2XHZic2m9YjUSE2ojNddO+4Qw/jnkP5jTr01MHa+0ougQKyFWE/kOFxsOZHPXmR0C3nZk9yZEh1hJz3VwxxerfDKK7E43b/25HYC7zuxAkCMDfrwTtsz0PsDm4v/vew38cJucbIKMUL7gDWg5QE44yzOZIaq5fAViDobo1oHHj0e39D/9yuORPjwz74E5k73bW5wMl7wHX9/gd7pQY3JMHYuZO+Gd06Ew07tt3hTY8D1c9bVMn6oBVrOJE5pF8uZVfchzuDAC6w9kc+OHy/02Nz+jcwKRwf+y39W2uQF3GXYvoNtJ4DYFKDMsYYuETT/BjDu9gdKoFnDh29B+OHQ6R3r5TB/jzc4JjYfzX5P3vC2cCI+HMSe35LEfN/h9irH9W2E1ayDnaBxTx6JSNcDhqpiRUxdTq2r1WGx3JoQnwIfnygAFkFKrAXfAFV9gQMu1lVKqPlX7U+zll1/O7bffzty5c3G5XLhcLubMmcMdd9zB6NGja3yBkyZNwmAwVPq1adMmXnnlFXJycnjggQdq9PkfeOABsrKySr/27NlTo4/f0JhNRppEBtMsOpiYUDnZiguzMfmCEyokmgAMbB9HswYw/SUh3MY9wzsCUu41a30Kt57ersLtmkQGcfewjoTYzBzKLQxYFjZ14U5Sc+yQtcc3iAOSPdH3WvhstDeIAxJg+XSUXK06WrYwOG1i4P29r5bsoPJ6XAa/TKo4ZWL3YsnIOftZOdFtxI6ZY7GoEOa/5BvEKXFog0w6q2FRIVaaRQXTJCqYzknhxIVVDKREBJm5ekCrfz/RzVjJ/Q0GPBhYlW6B+E7+b9Osj/xspo/xHX2buVt643S7VEoUF77sDeKANBWfPlbKrQCDwcBZJzShV4uoCk9xQc+mdEwKr/5rU8AxdCwqVUMcRa5yPXLqJiOnVo/FJt3gk1HeIA7I79y/noPcQ7iDtUeOUkrVp2pfep08eTI7d+5k6NChmM1yd7fbzdixY2ulR87dd9/NuHHjKr1NmzZtmDNnDosWLcJm8z1B6du3L1deeSXTpk0jKSmJgwd9sxlKvk9KSvL72DabrcJjHo+6NYvkq/EDeHLmBlbsyiQ6xMJ1p7bm0r7NifVzUljXrGYTF/VqRpOIIJ79dTOfL9vD7UPb8eVNJ/Ptyn0cznUwvEsi/dt5A097MwoCPp7d6abI5YFtcyru7HwOrPnCfx8btwsWvAznvAiWoKN7MXGdYOSL8Osk71hycxCc8z8Jxjj8ZEY17QXznvb/eDvnw4hnJUjUiB0zx2JBOmz+KfD+1Z9BhxHSZ6kWNIsO4cvx/Xl97la+WbkPl9vD8K5J3D2sA82jK5akVlubIQF3OVudztzdRXyyKpPel7xF8g+X+maYhcbDRe/CzHsDPIBdStKWBmhQ7yyUjLn+EwBIigzizav6sG5fFtOX7yXIYuSKfi1oGx/mN5ilquaYORaVqiEOlwdLmdKquppaVavH4uaZ8jvVn/nPY2hxMoRE1c5zK6WUOqJqB3KsVitffPEFkydPZvXq1QQHB9OtWzdatmxZG+sjPj6e+PgjZxK8/PLL/Oc//yn9fv/+/QwfPpwvvviCfv36AdC/f38eeughioqKsFikE//s2bPp2LGj37Iq5RVqM9OnZTTvXX0iBQ4XRgPEhwdh8tNItb5EhVgZ0a0JfVtFU+TyYDYZSAgPok/LGFwej8/VMoD4Sk7kjAbpl4PVT/AjqhX8MyvwQlJWgyPv6AM5wZHSZLbdUMnuMRilrCQsUYI43S6FtdN97+M6QtPWovzK96u6YzCBpZKAiTWs8qyWGtA8JoT/O68rtw9tjweIDLYQavuXJVUlwhJh4P3w5zO+24OiSOn/KO99fojM/CKu+C6D58/6muaOnYTl/ENIs64YE7tK0+LD/suhAPAU9xcKZL9vj6nEiCASI4IY1DEeIwaMDeh3llLq2OBwun16bsnUqkZeenRoY+B96dsxeBp5M2ellGrkjvqTe4cOHejQIXAfkrrWokULn+/DwuQEvG3btiQnJwNwxRVX8Pjjj3Pddddx//33s27dOl566SVefPHFOl9vYxUdYqUmLtrXpvhw3wCK0WjASMWTt6TIIJrHBLMnvWJmzpldEokItkDb0ys+QW6KjGdOWeN/AbHt/PfIqQ5LsPTDiS4XILUEwfAnpZRq8RvSwPiEiyG2beWPFxT179ajak5ovJTm/fZ//vefeH2tB3IAgiwmmtRGWWRwJJw8HtoOkQbheYeg3ZkUdL6Eaz/ZU9ocfXd6PqM+3U18eDCfXn8D7ROLS50KMqVPVPZ+/4/vdkJcRzi4zv/+5v38btbGxkqp2uDxeIp75JTJyDHWTY+cWtWkh2Qf+xPXEUOA0eRKKaXqRpUCORMnTmTy5MmEhoYycWIl/TugQY/yjoyMZNasWUyYMIE+ffoQFxfHo48+yo033ljfS1P1IDEiiKnjTmLc1KU+wZwTW0Xzf+d2JSzIDIYkGPEM/Hy/944bvoeL34WNP/h/4FPvAmstRrvCEqDdGdCiv5Ry2cKl8WuHs2DLLxVv32EEhPofLa/qgdEI3S+FDd9VyB6hzzgJBDZ2ITHQsj807QkuB1jDKLK7uKKfkSd/2iBli4DVZOSeYR1IiiwTfA2OgsGTYNq5FR/XYIT4jjD0Ufj00or7beHQflitvCSllPLH4ZLMFJ8eOcVNBd1uT+PNAmx/pjTh9zPx0jPoPgxRLfzcSSmlVF2pUiBn5cqVFBUVlf5/Y9CqVSs8nopXQ7p3785ff/1VDytSDVHbhDC+Gj+AA1kFHM6xkxwdQkK4zdv3xxYOPS6HlqfAig8h54BMzEk8AS56B368w1u2ZAmR3jgxdXQibg31/n9wlDz3TLP0X/F4ZAx6x5HS6LjsiHNV/yKawujPJKtr1Sfyb9nnGohpC6HHUANJS3BpdlpEsIlL+yZzeqcEdhzOxWAw0DoulPhwG0GWchlISd3hrGfgt0e9faJsEXDR2/KzC46BkS/A7Ee9PaNi2sAlH0Bkch2+QKXU8c7hLAnk+I4fB3C6PVgbaSDHENkKz5hvMXxzI6TLNE+sYXgGT8IQIPNRKaVU3TF4/EU7VEDZ2dlERkaSlZVFREREfS9H1TW3y1v24rRLo9bs/RI4iWwm/UHM9dgEtDBLpvcUZkNQhJTxBEXW33pq0TFzLLrdEnTzNxbueFZUINNSsvdJ35zwJhCWBObidH5XkUyNy08FkxVCYiHcf9N6VbuOmWNRqaOQmmun739+4+4zO9C3VQwAC7am8urcrWx84iyCrbVfKluixo9Ftwsy90iTfqddPlOEHbufK5RSqjGpdo+ca6+9lpdeeonwcN/RrXl5edx22228//77NbY4pRqcsr1LzDZpQtyQ0ouDIvUDVmOjvVv8C9QnqoTJAlHN5UsppeqJvTgjx2r2HT8O4HS7gboL5NQ4owliWgGt6nkhSimlyqv2GcS0adMoKKjYHLagoIAPP/ywRhallFJKKaVUQ2cvcgH+e+Q0+slVSimlGqwqZ+RkZ2fj8XjweDzk5OQQFORtTulyuZg5cyYJCQm1skillFJKKaUaGruzYrNjY5keOUoppVRtqHIgJyoqCoPBgMFg8Dt23GAw8Pjjj9fo4pSqSdmFRaTm2Fm9NxOTwUj35Ejiw2yEBlW7wlApVU0HsgrYnZbPjrQ82sSF0iIm1HdalVJKNUJ2P82OS/obN/aMHJfbQ0pWAf8cyuVQtp3OTcJpEhVMXFg99gJUSikFVCOQM3fuXDweD6effjpff/01MTExpfusVistW7akadOmtbJIpf6tjDwH783fwWvztlLS3ttogAfP7sylfZOJCLbWz8IKMsCeK2OVwxKk74c6vriKpEG12wW2MAiOru8V1bgdqblc9e5S9mV6y3KbxwTz0bX9aBUXWsk9lVKqYSsprbKa/PXIabyBHJfbw5q9mXy+eBsXtLfSLsjAn2sP8XeKi6cu6kZSZHB9L1EppY5rVQ7kDBo0CIAdO3bQokULDDphRTUi6/Zn8ercrT7b3B74z08b6dMyml4t6jiQU1QIhzfBrIdg53wZc97nWuh3k0y/UseH7P2w5C1Y/h7Yc2TM/fAnIb4zWI6NbJXUHDs3fvi3TxAHYE96ARM+XcGH155ErF7dVUo1UqUZOWWaHRtLeuS4Gm8g51B2IaGFKTwaNJ3Qnz+Gonx6tBzIoQGPMGPFTq48pT3BVs1oVkqp+lLtZscbN25kwYIFpd+/9tpr9OzZkyuuuIKMjIwaXZxSNSG7oIjXygVxynpv/g4Ki6+o1ZnDm+DdoRLEATmJX/gSfDZaRiqrY19OCnx2OSz4n/z7A+xaIO+LQxvrdWk1KS3Pzj+Hcv3uW78/m7Q8Rx2vSCmlao6/HjklGTkuT+MN5NgKDtJu1jWErngLHHng8WDa+QdNpp/NyGZ5pOvvbqWUqlfVDuTce++9ZGdnA7B27VomTpzI2WefzY4dO5g4cWKNL1Cpf8vhdHMo2x5w//7MgtIPYnWiIEMycdzOivtS1sChTXW3FlV/Dm+GA6sqbne75P2Rf2wExvMclQdJC46wXymlGjK7s3hqlTMXcvYDZQI57jr8bFHDgtLWY0z1c1HB5SBuyX8J9uTX/aKUUkqVqnYgZ8eOHXTp0gWAr7/+mnPPPZennnqK1157jZ9//rnGF6jUvxUWZKZvq8B9R/q3jSXUaqq7BTnyvJk4/mzW4+i4sOWXwPt2LQCH/yyWxiYm1Fra+LM8k9FAVIj2hVJKNV72IgnWWNd+Dj8/AHhKS6uKGnFpVdCWHwPus+ycSygFAfcrpZSqfdUO5FitVvLzJQr/22+/MWzYMABiYmJKM3WUakiCLCZuHNjWpxFhiRCriUv7NsfsZ1+tMRilJ04gobF1txZVf0LiAu+zhsr75BgQG2rlkt7JfvddcWJznX6ilGrU7E43BsCUugnyUyF9W5mMnMYbyCEsPvC+oAis5jq8AKaUUqqCap8pnHrqqUycOJHJkyezdOlSRo4cCcCWLVtITvb/YV2p+tYiNpgvbjqZTkneAEr35Ei+Gt+f5OiQul1MaLw0Ng6ky3l1txZVfzqfE3hfn2vkfXIMCA+ycM9ZHRk/qA3BFvngH2I1ceuQttx+RgdCbdosUynVeNmdLqxmI4asXbJh7/JjYvy4scfogPvcJ96IITSxDlejlFKqvGp/gn711Ve55ZZb+Oqrr3jjjTdo1kwm7Pz888+cddZZNb5ApWqC1WSiV4toPr6+H9kFRRgMEBlsJSa0HsaOmyxw8njYPld64pR19nMQrlOrjgsRzWDki/DTXb7bk7pD/1vAXA/vzVqSEB7ExDM7cNXJLSkschFsMREfbtMrukqpRs/udGMxesBVBEERsG8FpuZyQaYxjx8nIhnOnAyzH/HZ7Gl2IsbeY8Gkv7+VUqo+VTuQ06JFC2bMmFFh+4svvlgjC1KqNsWF2RpGKUdEU7jyS2l4u2mmlFN1OR/Cm0JQJWVX6thhC4Puo6DVKbDhB8g7BB3PhoTOEJ5U36urcVazqe6z35RSqpbZi9xYDS7wGCChK6Rv844fb8yBnOBI6HM1tB8G67+FgkzofA6GuPbH5N8opZRqbKoUyMnOziYiIqL0/ytTcjul1BGEJ8lXm0H1vRJVX2zhEN8RBt1b3ytRSil1FOxOF1ZPkVyQCY4Ge86x0SMHIChSvhIeqO+VKKWUKqdKgZzo6GgOHDhAQkICUVFRGAwVR5B4PB4MBgMul46SVUoppZRSxz67040FJ9gipFF9UQFGjxM4BgI5SimlGqwqBXLmzJlDTEwMAHPnzq3VBSnVINhzpdQlPx0sIdJ4trIJDkodj+x5xcdJWvFxEgdhCfW9KqWUqjN2pwsLDrCGye9BwOTMA8Dpdtfn0mpG9gHIPyw9gELiICwRLEH1vSqllDruVSmQM2iQt/SjdevWNG/evEJWjsfjYc+ePTW7OqXqQ+4hmPc0rJgK7uIMs8SucOmHENuuXpemVIORewj++C/8/b73OEnoIsdJXPv6XZtSStURe5Ebi9shpbJWCeQYHTlAI8/IcTnhwCr48mrI2ivbzEFw+qPQ83IIianX5Sml1PGu2uPHW7duzeHDhytsT09Pp3Xr1jWyKKXqjasIlrwFy9/znpwCHFwPH14A2fvrbWlKNRiuIlj+Pix7x/c4ObQBPjwfsvfV39qUUqoOFTrdWDyF0sC+JCOnqCQjpxEHcrL3wrRzvUEcAGchzHoQdi+qv3UppZQCjiKQU9ILp7zc3FyCgjTVUjVyOSmw5E3/+7L2QPr2ul2PUg1RTgoses3/vux9kLq1btejlFL1pNBRhMVtlx45llAAjPZjICNnyy9QlO9/35zJkFvxoq5SSqm6U+Xx4xMnTgTAYDDwyCOPEBLiHSPrcrlYsmQJPXv2rPEFKlWnigrAkRt4f+pWaHVq3a1HqYbIWQD2SiYYpm7RaWxKqeNCQUEBVpxgjQCLDQxGTEXZQEzjzsjZuzzwvrSt4HLU3VqUUkpVUOVAzsqVKwHJyFm7di1Wq7V0n9VqpUePHtxzzz01v0Kl6pIlWBoWBgrmxLat2/Uo1RBZgqUfRPFV5wriOtTtepRSqp4U2u0EUyS/Ew1GsAQX98iJwd2YAznN+sDaL/3vi20HJqv/fUoppepElQM5JdOqrrnmGl566SUiIiJqbVFK1ZvwJOg3Hv56ruK+yGQN5CgFEJYE/W+FeVMq7otops2OlVLHjQJ7EZEGp/TIAbCGYSrOWGzUGTkdRsDvT/gvrxrysE7yVEqpelbtHjkffPCBBnHUsctkgZNuhD7XyJW1EgmdYez3ENG0/tamVENhskDfa6Hv9WA0ebfHd4Kx3+lxopQ6bhQWObHhLO2PgyUYY3Egx9WYx49HJsPVP/r+PjcHwZmToeWA+luXUkopoBoZOWUtX76c6dOns3v3bhwO3xrZb775pkYWplS9CU+EYf+BU26H/HQZJxoSB2EJ9b0ypRqOsAQ483EYcCvkp+lxopQ6LhUWubGaDGAsvvhjCcHgyMFoaOQZOSazlFdd/zvkpcq0wtA4CEsEiw43UUqp+lbtjJzPP/+cAQMGsHHjRr799luKiopYv349c+bMITIysjbWWC0//fQT/fr1Izg4mOjoaC644AKf/bt372bkyJGEhISQkJDAvffei9PprJ/FqobLFgYxbSC5LyR00ZNTpfyxhUFMaz1OlFLHrQIXWMxlMhMtIVCYhcloaNxTqwAMBsnIadIdkvtAdEsN4iilVANR7Yycp556ihdffJEJEyYQHh7OSy+9ROvWrbnpppto0qRJbayxyr7++mtuuOEGnnrqKU4//XScTifr1q0r3e9yuRg5ciRJSUksXLiQAwcOMHbsWCwWC0899VQ9rlwppZRSSjU2dpcBW5DBu8Fsg9wCjAYDTlcjD+QopZRqsKqdkbNt2zZGjhwJyLSqvLw8DAYDd911F2+//XaNL7CqnE4nd9xxB88++yzjx4+nQ4cOdOnShUsvvbT0NrNmzWLDhg18/PHH9OzZkxEjRjB58mRee+21CiViSimllFJKVcbuNmEtm5FjtoKzAJPRgNujgRyllFK1o9qBnOjoaHJyZORss2bNSjNeMjMzyc/309m+jqxYsYJ9+/ZhNBrp1asXTZo0YcSIET4ZOYsWLaJbt24kJiaWbhs+fDjZ2dmsX7/e7+Pa7Xays7N9vpRSdU+PRaUaBj0WlRJFLjdOjL6BHJMNiiSQU9s9cvRYVEqp41e1AzkDBw5k9uzZAIwaNYo77riDG264gcsvv5yhQ4fW+AKravv27QA89thjPPzww8yYMYPo6GgGDx5Meno6ACkpKT5BHKD0+5SUFL+PO2XKFCIjI0u/mjdvXouvQikViB6LSjUMeiwqJQqLXABYrRbvRrMNnHZMhtrvkaPHolJKHb+qHch59dVXGT16NAAPPfQQEydO5ODBg1x88cW89957Nb7ASZMmYTAYKv3atGkT7uIRjw899BAXX3wxffr04YMPPsBgMPDll18e9fM/8MADZGVllX7t2bOnpl6aUqoa9FhUqmHQY1EpUegoDuRYygRyTFZwOzEaqfUeOXosKqXU8avazY5jYmJK/99oNDJp0qQaXVB5d999N+PGjav0Nm3atOHAgQMAdOnSpXS7zWajTZs27N69G4CkpCSWLl3qc9+DBw+W7vPHZrNhs9mOdvlKqRqix6JSDYMei0qJwnxpNWCzWL0bTXJsmAzgKr7IWFv0WFRKqeNXtQM5JpOJAwcOkJDgO2Y2LS2NhIQEXC5XjS0OID4+nvj4+CPerk+fPthsNjZv3sypp54KQFFRETt37qRly5YA9O/fnyeffJJDhw6Vrn/27NlERET4BICUUkoppZSqTGGOlO5bywZTTBLUMeGp9R45Simljl/VDuR4AnTgt9vtWK1Wv/vqQkREBOPHj+f//u//aN68OS1btuTZZ58FpJcPwLBhw+jSpQtjxozhv//9LykpKTz88MNMmDBBr2gopZRSSqkqK8jJAMBiDfJuNMtnYaPBU+s9cpRSSh2/qhzIefnllwEwGAy8++67hIWFle5zuVz8+eefdOrUqeZXWA3PPvssZrOZMWPGUFBQQL9+/ZgzZw7R0dGAZBPNmDGDm2++mf79+xMaGsrVV1/NE088Ua/rVkoppZRSjUthnkyJ8peRY0QDOUoppWpPlQM5L774IiAZOW+++SYmk3fUotVqpVWrVrz55ps1v8JqsFgsPPfcczz33HMBb9OyZUtmzpxZh6tSSimllFLHmvzcbCCYoKBg70ZzSY8cLa1SSilVe6ocyNmxYwcAQ4YM4ZtvvinNclFKKaWUUup4U5CfCwRjK9taoDiQoxk5SimlalO1e+TMnTvX53un00lhYaFPqZVSSimllFLHsvz8XCCeoLKfpkubHbs1I0cppVStMVb1hj/++CNTp0712fbkk08SFhZGVFQUw4YNIyMjo6bXp5RSSimlVIOTX1CACRdmo8G70WgGgxEjrlofP66UUur4VeVAzgsvvEBeXl7p9wsXLuTRRx/lkUceYfr06ezZs4fJkyfXyiKVUkoppZRqSAoK7QQZXL4bDQYwWzFqRo5SSqlaVOVAzvr16xkwYEDp91999RVnnnkmDz30EBdddBHPP/88P/74Y60sUimllFJKqYYkr7AIm9FVcYfJhsnj1h45Simlak2VAzk5OTnExsaWfj9//nyGDh1a+n3Xrl3Zv39/za5OKaWUUkqpBqjAUUSQ0U/5lNlWXFqlgRyllFK1o8rNjps1a8bGjRtp0aIFubm5rF69unQkOUBaWhohISG1skilAsmzO0nNtbM/swCbxURSRBCJEUGYytarK6UahfQ8O6m5Dg7n2IkJtRIfZiMu3Fbfy1JKKb/yi9zYTH6CNSYrxqLGH8hxuT0czC4kJbsQe5GLplHBxIfZCLFVe1aKUkqpGlbl38SjRo3izjvv5MEHH2TmzJkkJSVx8sknl+5fvnw5HTt2rJVFKuVPWp6d9//awZt/bi/9sBQdYuHNq/rQu2U0FlOVE87qVu5BcLsgKAqsGvxUCmB/ZgETp69i8fZ0AAa0iebG3uGc1DqakMj40pG+SinVILjd5DvBFuTnwpHJisnjatQ9chwuFyt2ZfLmvK2c385CiMXAD38XER8dxbgBrYgJtR75QZRSStWaKgdyHn30Ufbt28ftt99OUlISH3/8MSaTqXT/Z599xrnnnlsri1TKn7+2pPLavG0+2zLyixj7/lJm3TWQlrGh9bSyAHJSYNNPsPh1KMyCtkNh4N0Q3QZMenVLHb+yC4p45Lt1LN6eTkSQmY9GNafV/hlEzv8Y5hXi7nwexv63QnRLaSSqlFL1rTCTAo8Vm9l/IMeIE5er8U6tSsksxJx3kBdbLSF61QfgyGVg62Ecbj2eNXvSGdwpqb6XqJRSx7Uqnz0GBwfz4YcfBtw/d+7cGlmQUlVxOKeQ//22xe8+u9PN7A0Huf60NnW8qkrkHoLvboFtv3u3rfkcNn4PN8yBhC71tzal6llanp05mw8B8Ob5Ten2x40YD60t3W9c9g6s/xqunwsxrepplUopVUZ+OnmeIGxmP9m/Jismt7NRZ+S4cw7Sc/FdmPctLt0WvPZjWmz5HsulP5OaE62lr0opVY/+Ve3J008/TWZmZg0tRamqc7o87MkoCLh/44HsOlxNFaTv8A3ilCgqgNn/B4UNbL1K1aE8uwuPB1rGhtCuaJNPEKdUfjosfg2c9rpfoFJKlZefSj5B2MymivvMNoyexh3Iicrd6hPEKWXPIWb5i1jcgT+DKaWUqn3/KpDz1FNPkZ6eXlNrUarKrGYjHRLDAu4/sVVMHa6mCjb+GHjf1tlg10COOn6FB5kxGQ2c3DKChG1fB77hhu8loKOUUvUtP418bNgsfpLbjRaMHmejbnYc8c+3AffZts4kxJVbh6tRSilV3r8K5Hg8jfcPlGrcYsNs3H9WJ7/7IoLMnNo+ro5XdASVNTU2WQHt+6GOX3FhNi7u3YwilweXMSjwDc1B2iNHKdUwFAdygqx+AjkmCyZPUaMO5GCtpM+gOQiLv5IypZRSdabKv4WfeOIJ8vPza3MtSlVLrxbRPH1xNyKCvB+i2iWE8cVN/WkWFVyPK/Ojy/mB93W/HEJi624tSjUwoTYz9wzrSFRoEIc6XRX4hn2vhdCEuluYUkoFkp9GHiEEWfz1yLFgauSlVYaeVwTc5+41FkLj63A1Simlyqtys+PHH3+c8ePHExLizSzYsGEDTZs2rZWFKXUkkcEWLumdzMD28WTkObCYjMSEWhtm872IZnDaPfDXc77bo1rAwIlgqSQLQanjQEJEEHcP74QnLxZXt9GY1n7ue4PEbtD9UjDqVWClVAOQl0qupyXBflrkYLJi9BRR1IinVhmiW8FJN8LSt313xLbF2O9GMFnqZV1KKaVElQM5/sqomjdvXqOLUaq6zCYjTaOCadrQMnDKC46CAbdCx7Nh+buQlwpdL4TWAyEyub5Xp1SDEGozg60JDP8P9BkLy94DZz70vBKa9YEIvXCglGoYXDmHKMBGsMX/+HGT2964S6tCYmDwJOh2CSx9D+yZ0O1SaNEfIpvV9+qUUuq4V+VADoBBexModfSCoyG5DzTtCW4nmBtg5pBSDUFYvHw17wceN5it9b0ipZTykZeTCUCQv0/SJism8nC6XHW6phoXEitfzfqA26WfW5RSqgGpViCnQ4cORwzm6BQrpY7AaJIvpVTlTNX6E6WUUnUmL0emTQab/WXkWDDhxtmIS6t8GM3ypZRSqsGo1m/lxx9/nMjIyNpai1JKKaWUUg1ebl4eAMEBM3LcjT8jRymlVINVrUDO6NGjSUjQiSFKKaWUUuo45XaTW2gHIKjSjJxG3CNHKaVUg1bl8R/aH0cppZRSSh33CjPJc0vvrhC/GTnFgRz3MVJapZRSqsGpciDH39QqpZRSSimljiu5h8hFpmX6z8ixYsKlGTlKKaVqTZVLq9x6VUEppZRSSh3v8g6TSxBQeY+cRj1+XCmlVINW5YwcpZRSSimljnt5h8j1BGMxgtkYuEdOkQZylFJK1RIN5CillFJKKVVVeankGsIJsQTYb7JIaZUmsyullKolx1QgZ8uWLZx//vnExcURERHBqaeeyty5c31us3v3bkaOHElISAgJCQnce++9OJ3OelqxUkoppZRqVHIPkm2OJtRffxwAowWTwY22yFFKKVVbjqlAzjnnnIPT6WTOnDn8/fff9OjRg3POOYeUlBQAXC4XI0eOxOFwsHDhQqZNm8bUqVN59NFH63nlSimllFKqUchJIccYRYglQCDHYMBkMOLBoH1ylFJK1YpjJpCTmprKP//8w6RJk+jevTvt27fn6aefJj8/n3Xr1gEwa9YsNmzYwMcff0zPnj0ZMWIEkydP5rXXXsPhcNTzK1BKKaWUUg1e9gGyDeH+R48XM5kkyKMjyJVSStWGYyaQExsbS8eOHfnwww/Jy8vD6XTy1ltvkZCQQJ8+fQBYtGgR3bp1IzExsfR+w4cPJzs7m/Xr1/t9XLvdTnZ2ts+XUqru6bGoVMOgx6I67uXsJ4swggNl5AAmo3zErs0R5HosKqXU8euYCeQYDAZ+++03Vq5cSXh4OEFBQbzwwgv88ssvREdHA5CSkuITxAFKvy8pvypvypQpREZGln41b968dl+IUsovPRaVahj0WFTHvdyDZHlCCA3U7Ji6CeTosaiUUsevBh/ImTRpEgaDodKvTZs24fF4mDBhAgkJCfz1118sXbqUCy64gHPPPZcDBw4c9fM/8MADZGVllX7t2bOnBl+dUqqq9FhUqmHQY1Ed15x2KMgg220L3COHMoGcWiyt0mNRKaWOX5VU9zYMd999N+PGjav0Nm3atGHOnDnMmDGDjIwMIiIiAHj99deZPXs206ZNY9KkSSQlJbF06VKf+x48eBCApKQkv49ts9mw2Wz//oUopf4VPRaVahj0WFTHtRzJ4M5xmQkJNLWKsj1yai8jR49FpZQ6fjX4QE58fDzx8fFHvF1+fj4ARqNvkpHRaMRdfDWkf//+PPnkkxw6dIiEhAQAZs+eTUREBF26dKnhlSullFJKqWNKTgoeD+Q4TUcorTIBtRvIUUopdfxq8KVVVdW/f3+io6O5+uqrWb16NVu2bOHee+9lx44djBw5EoBhw4bRpUsXxowZw+rVq/n11195+OGHmTBhgl7RUEoppZRSlcs5gB0LRW5D5aVVppIeOTq1SimlVM07ZgI5cXFx/PLLL+Tm5nL66afTt29f5s+fz/fff0+PHj0AMJlMzJgxA5PJRP/+/bnqqqsYO3YsTzzxRD2vXimllFJKNXg5KWQZZYhG5YEcycgpqsVmx0oppY5fDb60qjr69u3Lr7/+WultWrZsycyZM+toRUoppZRS6piRvZfM4GQogHBr4JuZiwM5Li2tUkopVQuOmYwcpZRSSimlalXWXjKtTQAIqyQjx2iSa6VFWlqllFKqFmggRymllFJKqarI3E2mJRGA0CqUVmmzY6WUUrVBAzlKKaWUUkpVRdZessxxAJVPrSrOyHG5NSNHKaVUzdNAjlJKKaWUUkfitEPuQTJN0YRawGSsJCPHLFEebXaslFKqNmggRymllFJKqSPJ3gdAJhGV9scBMJdk5BQ5an1ZSimljj8ayFFKKaWUUupIsvYCkOkJPWIgx2gubnbsKKj1ZSmllDr+aCBHKaWUUkqpIykJ5LiCKu2PA2AyyQ2cdg3kKKWUqnnm+l6AUkoppZRSDV7GLgiOJt1uIMxW+U1NxRk5Toe9DhamlFLqeKMZOUoppZRSSh1J+nYIb0J6oYdwa+WlVSazFQCno7AuVqaUUuo4o4EcpZRSSimljqQ4kJNRhUCOuTiQ4yrSjByllFI1TwM5SimllFJKHUn6djxhSWTYPYRbK7+psWT8uAZylFJK1QIN5CillFJKKVWZwmwoSCc3JBmnG8KPMLXKZCludqw9cpRSStUCDeQopZRSSilVmYwd8h9rE4AjllYZzDbMOClyOGp9aUoppY4/GshRSimllFKqMunbAUgzJwBHDuRgNGHGhcNZVNsrU0opdRzSQI5SSimllFKVSdsKtggyXMEAR+yRA2DBhcOhgRyllFI1TwM5SimllFJKVebwZohqTmqBB4AI2xEycgCLwU2RZuQopZSqBRrIUUoppZRSqjKHN0FEMqkFHsIsYDYeOZBjNrhxFDnrYHFKKaWONxrIUUoppZRSKhC3C1K3QFQL0go8RFYhGwdKAjmuWl6cUkqp45EGcpRSSimllAokczc47RCZTGqBm4gjNTouZjZ4cLg0kKOUUqrmaSBHKaWUUkqpQFK3yH+jWpBa4KlSfxyQHjkOp7sWF6aUUup4pYEcpZRSSimlAjm4HiwhEBJXrUCO2eChyKWBHKWUUjVPAzlKKaWUUkoFkrIWYtqAwcDhfE/VS6uMBs3IUUopVSs0kKPUscBVJM0YlWps3E5w6VQXpVQDlrIGolvjdHvIKPQQHVTVQA44XJ5aXlwdcLvlc4ZSSqkGw1zfC1BK/QvZ+2DPUljzBdgi4cTrILYthMTW98qUqlzuIRnnu/x9OUHoPRaa9IDwpPpemVJKeTnyIG0bdBxJaoEHD1Q5kGMyGihyNOKMnPx0SNsKy94DexZ0uxSa94PIZvW9MqWUOu5pIEepxiprH3x0gbcJI8Caz+GkG2HQJAjVYI5qoHIPwcx7YMP33m2bZkByP7h0GkQ0qb+1KaVUWQc3AB6IacPBPMmuqWogx2I0YHdV7bYNTn4a/PEsLHnDu23zzxDXHsZ8B5HJ9bY0pZRSjai06sknn2TAgAGEhIQQFRXl9za7d+9m5MiRhISEkJCQwL333ovT6ZuyP2/ePHr37o3NZqNdu3ZMnTq19hevVE1zFcGyd32DOCWWvg2Zu+p+TUpVVcoa3yBOib1LYMsvdb8epZQK5MAqMJohqgWH8iW7JrqqzY5NBhxuQ+Msfc7c7RvEKZH6Dyx9R0utlFKqnjWaQI7D4WDUqFHcfPPNfve7XC5GjhyJw+Fg4cKFTJs2jalTp/Loo4+W3mbHjh2MHDmSIUOGsGrVKu68806uv/56fv3117p6GUrVjLzDsGJa4P0rP6q7tShVHUX5sOStwPuXvQN5qXW3HqWUqszeZRDTFkwWDuZ7MBogwla1u5pNRhxYwJFbu2usDSs/CbxvxTT5HKKUUqreNJrSqscffxwgYAbNrFmz2LBhA7/99huJiYn07NmTyZMnc//99/PYY49htVp58803ad26Nc8//zwAnTt3Zv78+bz44osMHz68rl6KUv+exwNOe+D99py6W4tS1eF2Q1FB4P1FBeBpxD0llFLHlj1LIekEAA7muYm2GTAaqlpaZcSBWf4mB0XW5iprniMv8D5noXwOUUopVW8aTUbOkSxatIhu3bqRmJhYum348OFkZ2ezfv360tucccYZPvcbPnw4ixYtCvi4drud7Oxsny+l6l1wFHQ8O/D+HpfX2VLqih6LxwhbGHQfHXh/14sgOKbu1qOqTY9FddzIS4OMHRDXCYB9OR7igqve88ZsNkkgp7B2jpFaPRa7jQq8r+NICIqquedSSilVbcdMICclJcUniAOUfp+SklLpbbKzsyko8H+FeMqUKURGRpZ+NW/evBZWr1Q1WUNh0P1gi6i4r1lfSOxa92uqZXosHkPaDoHYdhW3hyXI9CpTo0kWPS7psaiOG/uWy3/jOwKwN8dNbEg1Ajkmk5RW1VKWbK0ei4ldIPmkittt4TB4EthCa+65lFJKVVu9BnImTZqEwWCo9GvTpk31uUQeeOABsrKySr/27NlTr+tRqlRMG7hxHvQaC6FxENUShv0HLvv4mBzhrMfiMSSyGYz9XqarRTSVAM7Jt8B1v0F0y/penToCPRbVcWPnfAiJhTC5CLg31129jByTiSKPGey1k5FTq8dieBJc+iEMexKiW8nPodcY+dwR06bmnkcppdRRqdfLnnfffTfjxo2r9DZt2lTtj0VSUhJLly712Xbw4MHSfSX/LdlW9jYREREEBwf7fVybzYbNVsWudkrVJaMRYtvC2f+FIQ+AwQSh8bL9GKTH4jEmMhkG3gd9x4EHCIkBs/77NgZ6LKrjxo4/IKkbGAw43R4O5lWvtMpiMRf3yKmdQE6tH4sRTSTI3m2U9C4LigKr/8/LSiml6la9BnLi4+OJj4+vkcfq378/Tz75JIcOHSIhIQGA2bNnExERQZcuXUpvM3PmTJ/7zZ49m/79+9fIGpSqF5Zg+VKqsTGZILxJfa9CKaUqKsiAA2tgwG0AHMj14PJAfEjVL5aYTSaKarFHTp0wGiE88ci3U0opVacazaX73bt3s2rVKnbv3o3L5WLVqlWsWrWK3FwZ6Ths2DC6dOnCmDFjWL16Nb/++isPP/wwEyZMKL1aMX78eLZv3859993Hpk2beP3115k+fTp33XVXfb40pZRSSinVkOxcAHggqTsAWzNdADQLq05plaFWe+QopZQ6fjWajpKPPvoo06ZNK/2+V69eAMydO5fBgwdjMpmYMWMGN998M/379yc0NJSrr76aJ554ovQ+rVu35qeffuKuu+7ipZdeIjk5mXfffVdHjyullFJKKa8tv0Bk89Kec1sz3NhMEFud0iojkpGjgRyllFI1rNEEcqZOncrUqVMrvU3Lli0rlE6VN3jwYFauXFmDK1NKKaWUUscMtxu2/AotTyndtDXTTbMwA0ZDNTJyjAacmHAVZmOqjXUqpZQ6bjWa0iqllFJKKaVq3YGVkHcImnvHb69PddEsvHofm4OKozcFBfk1uTqllFJKAzlKKaWUUkqV2vA92CIgoTMABUUeNqW5aR9dvY/NNrNk7+TnayBHKaVUzdJAjlJKKaWUUgAeD6z9SsqqjNKBYG2qC6cH2kdXr0DKVpKRk59X06tUSil1nNNAjlJKKaWUUgB7lkL2Pmg9sHTT/L1OwizQIqLq/XHAG8jJy8utyRUqpZRSGshRSimllFIKgJUfQVgiJHYt3TR3t5Nu8aZqNToGb2mV9shRSilV0zSQo5RSSimllD0H1n0N7c4Ag3xEPpjnZm2qm14J1Z87VdLsOL/QAS5nTa5UKaXUcU4DOUoppZRSSq3+HJyFEsgpNnuXE6MBeiVWP5BjMxU3O8YGBek1tkyllFJKAzlKKaWUUur45nbD4teh5QAIjS/dPHtnEZ1jjYRZq1dWBRAkvZIpwAZ5h2tqpUoppZQGcpRSSiml1HFu04+Qvh06n1+6Kb/Iw6J9LnofRVkVgMUIBjzkezSQo5RSqmZpIEcppZRSSh2/XE74fTI07QMJnUs3L9rvxOE+urIqAIPBQJDJIKVVeak1tVqllFIKc30vQCnVCOSnQ84B2LUQrGHQop9M9bCG1vfKlBLOIsjZD/tXyXs1uS9EtYCwhPpemVKqoVvyJqRthXNe9Nn8xx4niSEGkkKrX1ZVwmY2UOAJbZwZOY58yD0Ie5ZII+gWJ0N4UwiNre+VKaXUcU8DOUqpyuUegl8fgrXTvdsMRjj3Jeh6IdjC629tSoEEcfYsgk8vhaIC7/ZmJ8JlH0JE0/pbm1KqYUtZB3OegM7nQWw7n11/7JGx44Zqjh0vK8gM+e5I+VvamDhyYdNM+G48uF3e7Z3Pg7Ofg/DE+lubUkopLa1SSh3Bll99gzgAHjf8cBtk7qmfNSlVVs5++GSUbxAHYN8y+ONZKCqsn3UppRq2zN3yuyOyOfQe47NrZ5abXdkeeiT8u4/KNhMUmMIaX0ZO1j749kbfIA7Axh9gw3fg8dTLspRSSgkN5CilAss9BAv+F3j/ig/rbClKBbTvbxkZ7M/qTyCvkV0JV0rVvoyd8MHZgBuGPAzmIJ/dc3YXYTFC17ij649TIshsINsQLiVKjcma6YGDNQtfbnyvRymljjEayFFKBeZ2Vn4VMWu33Eap+pS9L/A+px1cRXW3FqVUw5e6Fd4/CzwuGD4FQuMq3OTXHU66xhkJNh99WRVAhNVAGtHSg6cxydwVeF/e4YqZOkoppeqUBnKUUoFZw6F5v8D72w8Ho7baUvUs+cTA+6JaaFNupZRX2jaYOhJMFhj+NITGV7jJ/lw3Sw+4ODHp3/99i7AZSPWEQ8YucDr+9ePVmQ7DA+9LPlF/ryqlVD3TQI5SKrCgcDj9ETD6SS0PS4C2p9f9mpQqL7o1NOnhf9+w/0B4Ut2uRynVMKX+I+VUJgsMexJCYvze7LONDmwm6N/s35VVAUTaIM0ZJNk/GTv+9ePVmRb9/TeKNxjhzCcgOKrOl6SUUspLAzlKqcrFtYerf4KELvK9wQDth8E1P0NU8/pdm1Ig01NGfwY9r5ITNJDmpaOmQutB9bo0pVQDsXe5lFOZbRLECY72e7OtGS7eXePgjFbmf11WBRBpNZDmMEm7mdR//vXj1ZnIZBg3EzqeLcEbgPiOMPYHiO9cv2tTSiml48eVUkdgCYaW/eXDmz1bsnOCYyAoor5XppRXZDMY+SwMug/cRWAJhYgm9b0qpVR9KyqExa/DvCkyXnzIQxAUWeFmTreHd9Y4eHWFnfhgAxe2t9TI00fYDBS5DeRY44lI3VIjj1lnYlrDRW9Dfpr0w7NFSDauUkqpeqeBHKVU1YTFy5dSDZUlBKJb1vcqlFL1zeWE3Ytg80yZvlSQDp3Ph95jvVl7ZTjdHm6eVcDvu50Ma2Xmko4WQiz/PhsHpNkxQFpIu8YXyAGwhcuXUkqpBkUDOUoppZRSqvE7uB7+ngprv5LgTUgstDwFOo2EiP9n777jo6jTB45/tvdN74VA6L0jClYQe2+IimIvpx7eefqzt1Pv1PM8O4hdsXdFBUQFadJBektCettsNtt3f38MJITshmIqPO/Xi1eS+c7OfmfJTGae+X6fJyPqy/69xMucvAB3jDAwNOXP58XZW4JJCeTkWQfQdesXEAqBWjIbCCGE+HMkkCOEEEIIITqHoB8CXqViYjgEjgLY8Qus+gAKlihTf7sdDzljlKlUquaDJp9v9vPKKh+T+upaPIgDkGxWYdHBWv1gjqt9FQqWQnYz1SCFEEKIAyCBHCGEEEII0bGEw+BxKJWedi2Dnb9B/hIlcEO48boqDaQPhuPuguyjlCDPPgKhMD/nB5hfEKS0LoReo8LhDTE3L8ixmRpO79Y6l8QqlYqcGDWr6hKUBMurP5BAjhBCiD9NAjkHKRxWLh5qamrauSdCHJ5sNhsq1f5zE8ixKETrarNjMRxWElSHgsrPaq3y7wDeu0232VZCQcIBLw5PAKc3hN5gJN6iR6dpwek4oYAysgWUESsafdPPZvdn6Pf7qagLEAirsZuM2Ezaxr8X4TD4nKidRaiq86ipKKSi2kG5O8xmt5XNnhiCKj1drUF62Tx0NTpJ0zjQBr0Q8OL0+HHUedB7ykjwFmD0VaEKeCDoRRUOEQ7DLpKYZTiFn0K349ebGB3v5JTEMnJMXjDHEYrrRlhrwuWHqjIvobAXjQrcASiqC7OiNMwX28KU1EGKGRKNEAiDRgWX91ZxfGYIr9fbcp/vPnrGhPl2R4gtfc+n++/Tccf2xN/3fKWC1gGQv4tCdAwHeiwK0RZU4T1nfXFACgoKyMqSkstCtBaHw4Hdvv+KWHIsCtG6WuNY3HCzhV6JLT995XDgCesY5X0BB9b27sphzR52tvl7ujHiVylJlu/RvsO12m8BuH2Wh/8u9u339fJ3UYiO4UCPRSHaggRyDlIoFKKwsJBwOEx2djb5+fkd/oCuqakhKyurU/QVpL+traP390Cfduw5Fvddv6PvX2s7kvdf9r1l9/3PHovt5Uj+PWhJ8jm2jJb4HNv7WJTfhYMjn9fB6UyfV0f5OycEyNSqg6ZWq8nMzKwftmq32zv8SWePztRXkP62ts7W333tORaj6ez792cdyfsv+962+76/Y7G9HMm/By1JPseW0RafY2sfi/K7cHDk8zo48nkJcXCk/qEQQgghhBBCCCFEJyGBHCGEEEIIIYQQQohOQgI5h8hgMPDAAw9gMBxYxYH21Jn6CtLf1tbZ+nuwDvf9258jef9l34/Mfd+XfBYtQz7HlnE4fI6Hwz60Jfm8Do58XkIcGkl2LIQQQgghhBBCCNFJyIgcIYQQQgghhBBCiE5CAjlCCCGEEEIIIYQQnYQEcoQQQgghhBBCCCE6CQnkCCGEEEIIIYQQQnQSEsgRQgghhBBCCCGE6CQkkCOEEEIIIYQQQgjRSUggRwghhBBCCCGEEKKTkEDOQQqHw9TU1BAOh9u7K0Ic0eRYFKJjkGNRiI5BjkUhhDhySCDnIDmdTmJiYnA6ne3dFSGOaHIsCtExyLEoRMcgx6IQQhw5JJAjhBBCCCGEEEII0UlIIEcIIYQQQgghhBCik5BAjhBCCCGEEEIIIUQnIYEcIYQQQgghhBBCiE5CAjlCCCGEEEIIIYQQnYS2vTsgREWtl50VdXy1qhCVCs4alE5WgpkEi6G9uyaEaEVuX4Bih4fv1haTX1XHsT2SGJQVS3qsqb27JoQQh8TjD1Lk8PD92mJ2VroY0z2RIdlxcl4TQgjRoiSQI9pVmdPLA1+u5ds1xfXLZizYwXlDMvi/0/qQaJNgjhCHI48/yLxNZdz87nJCYWXZ+0vyyYg18f51R5Edb27fDgohxEHyBYL8urmcG95ZRnD3ie39Jfmk2o3MvO4ochIt7dxDIYQQhwuZWiXa1bKdVY2COHt8umIXa3Y52qFHQoi2UOr0cuv7K+qDOHvsqnbz6Nd/UOsJtE/HhBDiEJXUeLnlveX1QZw9ims8PPjlOmrc/nbqmRBCiMONBHJEu6lx+5n+67ao7dPnb6PWKxc9QhyOVuVX4w+GI7bNXl9CZZ2vjXskhBB/zh9FNXgDoYhtP28uo0rOa0IIIVqIBHJEu/EHQ9R6oz91d3oCBKLc6AkhOjdHM0+mQ2EIBCPfDAkhREfV3IibcJiowetOIxyGZW+CR0ZMCyFEe5NAjmg3MSYdJ/dNidp+av80bEZdG/ZICNFWhnaJi9rWNdGC1Sgp3IQQncvAzNiobZlxJmyd/bxWtQO+uhW+v6e9eyKEEEc8CeSIdqPVqLlweBZx5qbBmiSrgTMHpaFRq9qhZ0KI1pZqN0QM5KpU8NBZ/Ui2GduhV0IIceiSbQbOHJgWse3hs/qRYu/k57XyTY2/CiGEaDcSyBHtKivezGc3HcN5QzLQa9QYtGouHJbJJzeOJjNOqtYIcbiKtxh49Jz+/N9pfUiyGlCpYEh2LB9dP5phzYzWEUKIjirOouf+M/tx3xl9SLYp57WBmTF8cN1RjOyW0N7d+/PKNihfq3Yq06yEEEK0G1U4LGfig1FTU0NMTAwOhwO73d7e3Tls1PkCOOqUueWxZh0mfScffixanRyLh4dQKExprZdQKIxJryHOrG/vLomDJMeiEI2FQmHKar0EQ2GMOg3xlrY5r7X6sfj5TbDyXeX7O7eDOb7l30MIIcQBkbtl0SGY9VrMErwR4oijVqtI7ezTDYQQYi9qtarzT6OKpHwTWFOgtgTqKiSQI4QQ7UimVgkhhBBCCCGaV1cJMVm7v69o374IIcQRTgI5QgghhBBCiOZ5HGDbncxZAjlCCNGuJJAjhBBCCCGEaJ63RplaBRLIEUKIdiaBHCGEEEIIIUR0AS8EfWC0gcEmgRwhhGhnEsgRQgghhBBCROepUb7qLGCIkUCOEEK0MwnkCCGEEEIIIaLz7g7k6C3KqJy6qvbtjxBCHOEkkCOEEEIIIYSIzlOtfNVbdk+tKm/X7gghxJFOAjlCCCGEEEKI6OqnVplBb5epVUII0c4kkCOEEEIIIYSIbu+pVXqzUopcCCFEu5FAjhBCCCGEECK6vUfk6Mzgq23f/gghxBFOAjlCCCGEEEKI6Lw1oDOBWqN89UogRwgh2pMEcoQQQgghhBDReWqUaVXQMCInHG7fPgkhxBHssAjkvPDCC+Tk5GA0Ghk1ahRLliw5oNfNnDkTlUrFOeec07odFEIIIYQQorPy1YLWpHyvM0M4BP669u2TEEIcwTp9IOeDDz5g6tSpPPDAAyxfvpxBgwYxYcIESktLm33djh07+Nvf/sbYsWPbqKdCCCGEEEJ0Qn43aA3K97rdAR2vs/36I4QQR7hOH8h55plnuPbaa7nqqqvo27cvL7/8MmazmRkzZkR9TTAYZNKkSTz00EN069atDXsrhBBCCCFEJxPwgEavfK8zK18lT44QQrSbTh3I8fl8LFu2jHHjxtUvU6vVjBs3joULF0Z93cMPP0xycjJXX311W3RTCCGEEEKIzstf1xDI0e8J5NS0X3+EEOIIp23vDvwZ5eXlBINBUlJSGi1PSUlhw4YNEV8zf/58XnvtNVauXHlA7+H1evF6vfU/19TIHy0h2oMci0J0DHIsCtExtOmxuPfUKq1MrRJCiPbWqUfkHCyn08nll1/OtGnTSExMPKDXPP7448TExNT/y8rKauVeHp4qar1sKKph7oZS1uxyUFrj+VPbc9T5qHR5CYakYsKRQo7Fw4ejzsfWslrmbihl+c4qiqrdhPepfuL1B6l0eanzBtqplyIaORaFiK7OG2BnhYt5G0tZsKWcgqo6fIFgq7xXmx6LfnfTETk+mVolhBDtRRXe9+q5E/H5fJjNZj7++ONGlacmT55MdXU1X3zxRaP1V65cyZAhQ9BoNPXLQqEQoEzJ2rhxI7m5uY1eE+lpR1ZWFg6HA7vd3gp7dfgprHZzy3vLWZ5XXb8sN8nCa5NHkJNoOahtldZ4WLStgtcX7MATCHLmwHTOGpxOZpy5hXstOho5Fg8PZU4Pj3y9ni9XFdYvS7IamHHlCPql2/EHQ+ysrGPaL9tYVVBNdryZG4/PpUeyDbtJ1449F3vIsShEZNV1Pt5ZtJNnZ28msPtBk0mn4emLBnF8ryTM+pYdCN+mx+L0cWCMgWNuh6AP3jkPzn0FBl3Ssu8jhBDigHTqqVV6vZ5hw4YxZ86c+kBOKBRizpw53HLLLU3W7927N2vWrGm07N5778XpdPLf//434pMMg8GAwWBolf4fCWrcfu79bE2jIA7A1jIX17z1O+9fO4okm/GAtlXm9HDHh6v4dUt5/bL1RRt5a+FOPrphNFnxEsw5nMmx2Pn5gyHeXrizURAHoKzWy6Tpi/j2trHkV9Zx+WtL6m+CNpXUMnt9KY+c3Y8Lhmdh0mkibVq0ITkWhYhs+c4qnvphU6Nlbn+QW95bzje3jqVPWssGV9r0WPTXgSVZ+V6tA7VWplYJIUQ76vRTq6ZOncq0adN48803Wb9+PTfeeCMul4urrroKgCuuuIK7774bAKPRSP/+/Rv9i42NxWaz0b9/f/R6fXvuymGpwuVl7sayiG1bSmspc3ojtkWyodjZKIizR3GNh7cW7mi1octCiJZR5vTy+oIdEdtqPAFWFzj49/eb6oM4e3vk6/WUH8T5Qggh2lKly8vzP22J2BYKw/uL8wgGQ23cqxbk9zTkyFGpQG+RQI4QQrSjTj0iB+Diiy+mrKyM+++/n+LiYgYPHsysWbPqEyDn5eWhVnf6eFWn5fI2H1ypdPkOaDvBUIj3l+RFbf98RSFXj+lGaow8rReio/IFQjibyXmzvcxFIBT5RscXDLGjwiUj74QQHVKtJ0BBlTtq+/YKF3X+IDZNJ70m9dc1BHJASXgsgRwhhGg3nT6QA3DLLbdEnEoFMG/evGZf+8Ybb7R8h0Q9u1GLVq2K+IQdIMV+YNOqANQqVfRGFTTTKoToAIw6Nck2A6VRRtb0y7Dz+m/bo75eJUe5EKKD0mvV9E61RT+/pdsxdNYgDuxOdrxXIEdnVII7Qggh2kUn/osiOoNEm4FLRkSuojCmewKJ1gOb261Rq5k4Mjtq+/lDM4i3ytQ4ITqyFLuRqSf3jNiWGWeiW5IVkz7yqDqDVk2XBBmNI4TomBKtBq4/LpdIz5xMOg3nDM5A35lzfAU8oN3rOktrlKpVQgjRjiSQI1qVWa/ltnE9mDy6C/rdT6LUKjh9QBr/vnAQcZYDD770TLFxUu/kJssz40xcflQXdJ35SZcQRwCVSsXJfVO457Q+WA0NA0JHdo3j3WtGkR1v5pkLB6PTNL0TevjsfiTZJMGuEKJj0mrUdE008+8LBpG817kqN8nCy5cNJcHWiR82hUJKIGfvETlaI/hkRI4QQrSXw2JqlejYkmxG7jqtN1eP7YrTE8Ci15Jo02M1HFwp4SSbgcfPH8CKvGpeX7Adjz/EWYPTmdAvlYxYUyv1XgjRkuItBq48pgunDUjF4Q5g0KlJsOiJNSs3OYOyYvjutmN587ft+Nm8ewABAABJREFUrMx30CXezHXHdaNrogVjZ36aLYQ47KXHmjm2h5puScOocvnQqlXEWfRkxJhIOMARyB1SwKN81e4byHG1T3+EEEJIIEe0DZNOS3b8n/91S7YZmdAvlaNzEwiGwsSYdKiay50jhOhwdBoNGXFmMuKatum1GronW7nvjL7UeYMY9GpMOvlTJYToHJLtRpJsBhxuPxq1Cpvx4B5adUgRAzkGCeQIIUQ7kqtj0SkdFhdGQoio9FoNeq2MwBFCdD4qlap+lOFhYU9SY81e+6QzgbO4ffojhBBCcuQIIYQQQgghovDvLqu+74gcv4zIEUKI9iKBHCGEEEIIIURk9YEcY8MyrUmmVgkhRDuSQI4QQgghhBAisj2BnEZTqyTZsRBCtCcJ5AghhBBCCCEiC0QI5GiNDQEeIYQQbU4COUIIIYQQQojIAl7la6NAjgmCPgj626dPQghxhJNAjhBCCCGEECKyiIGc3YmPZXqVEEK0CwnkCCGEEEIIISKrD+RoG5bpTMpXCeQIIUS70O5/FSFaRpXLhycQRK9Rk2A17P8FQogOobrOh9sfRKtWk2STY1cIIZpz2F3vBDzKV7WuYZmMyBFCiHYlgRzR6pxuP2sKHfxr1kY2lTjJijNz+/gejOqaQLxFv/8NCCHahcsbYEOxkydnrWftrhrSYoz85cQejO2ReHjcnAghRAuqcftZu8/1zl/H92Rk1/jOfb0T9IJao/zbQ7t7RI5fAjlCCNEeZGqVaFXBYIjZG0q4dNpiVuZXU+cLsrHEyY3vLOet33ZQ5w20dxeFEBGEw2EWbq3g/Jd+Y8n2Kup8QbaWubj9g5X8b+4WajyS4FIIIfYIBEP8+EfT650b3lnGO4s6+fVOwNs4Pw4o5cdBRuQIIUQ7kUCOaFUlTi8PfvlHxLbnf9pCea23jXskhDgQJTVe7v18bcS2NxfuoEKOXSGEqFfq9PLQ1+sitj03ZwvlLl8b96gFBTxNAznaPYGcurbvjxBCCAnkiNZVVefD4Y785D4QClNQ7W7jHgkhDkSNx09xjSdiWzgMG4qcbdwjIYTouKrqfNS4I4+6CYTCFFZ14uudgA80usbL6gM5tW3fHyGEEBLIEa1Lq27+V8yo1TTbLoRoH1q1qtl2s0GOXSGE2GN/50yDrhNfcjc3IscvI3KEEKI9SLJj0ariLXpykyxsLWs6h9pu0pIaYzz0jbsqwF0BwQAYY8CWBvsJHAkhDkycWc/AzBhWFziatBl1anKTrO3Qq334XFBbqtxI6K1gS2m4uRBCiDYUZ9HTNdGC0+Pn1tEJHJ2uQhX0U+wzMm1lHSn2TnxuCngbV6wCJfGxRi9Tq4QQop1IIEe0qiSbgecuGcLFry6idq9EfzqNihcvHUbyoZYyLt0An98IhcuVn60pcOq/IPckMNpaoOdCHNniLHqevnAQF76ykOq6humRahX8b+LQQz92W0pNIcx+CNZ+BKGgEsAZeS0cfStYk9u3b0KII06yzcjLlw3D6txG2rw7UP+6FIBcSxIjx/8Ttb43YGrfTh6qYIRkx6Ccd6VqlRBCtAsJ5IhW1yfNzne3jeXH9SUs2V5J3zQbZw7KICPWiFZzCCNoqvPg9VPAXdWwrLYEPpoMV34DOWNarvNCHMG6J1v5+i9jmLexjPlbyumeZOXcIRmkx5nQt+e0yLpK+PIvsGV2w7KAB377nzJCb9wDoOukN0xCiE6rp7Ea1dvngau8YaGrDN3n18Lln0Huie3XuT8j4G2aIweUQI5UrRJCiHYhgRzR6tRqFVnxZqYc05UrR+eg3s888v3a9nPjIM7efrwfJn0M5vg/9x5CCFQqFZlxZi47qguXjsz+88duS3GVNQ7i7O331+CoGyAup027JIQQqryFjYM4e/vxfkgdCJbEtu1US4g0tQqUEuQytUoIIdqFBHJEmzqUG8FKl5eKWh8uX4C0GBPJ238m6laK14C/E1eGEKKDaq8gTnmtl4paL25/kDiznkSrAUvNrugvCPrA0zSvjxBCtLod86O3laxVRg52RgFP9BE5MrVKCCHahQRyRIeWX1nHrTNXsCKvGoBRXeN5vXsPzNFeEJMJavm1FuJwsK2slhveWcamEqW8rUatYtLILB4YEU+zE7t0ljbpnxBC7C2Y0CP6ucme0XmvTwLeyH3XGmRqlRBCtBMp8SM6rDKnl2vf+r0+iAOwZEclpVmnRr8YOvbvSuUaIUSnVuRwM2n64vogDkAwFOatRXls99ogITfyC3PHgSWpjXophBCKYoebXSknRh65AoTH3KEUZuiMAs0kO5apVUII0S4kkCM6rNIaDxuKnY2WhcPwzwVOys98C/R7PXVXqWDkddBjfBv3UgjRGraXuShyRJ6GcMNn+QQu/gDiujZuSBsMZz4LpphW758QQuxtZ0UdD/5cTdlZ74Le2qitbsAVeLqfplyrdEYBTzOBHBmRI4QQ7aGTjvEUnZnbH6TY4eGnDaUUOdyM7ZFEr1QbKXZjo/UKo9zE/bCxmlpfDK9duwBTzU7lIiK5F1iSwWhvi10QQhyEQDBEkcPDwm0VbC5xMiQ7jsFZsaTHRq8staWstpk2FwWadHKmfAeOXUop8rguYEuT0uNCiHaxrdzF3M3VfNurF+dM+RVV5TbwOtGk9GG900is30L39u7koQp4IheR0BrAXd3m3RFCCCGBHNHGPP4gP20o5Zb3lhMKK8um/bqd7slW3poystGNXXqMMcpWYMnOGsrUQ8jO7Rp1HSFE+wuFwqwucDBp+mLc/uDupdtJtOqZed1ouidbI76ue1Lk5QA2gxadRqUEbmxprdBrIYQ4ON0SLdx8Qne2VngY9PVObAYtBp2VCtdWxnZP5LFzO/GUz2C0qVUm8DeTfF4IIUSrkalVok2V1Hj4y/sr6oM4e2wpreXZHzfh9gXqlyXbjfROtUXczvlDM0i0Glqzq0KIFlBc4+Gat37fK4ijKK/1cdvMFVS6vBFf1zXRQqo9cjB3ypiuJNuiB3qFEKKtdUkw0z/DzlsLdwLg9AYor/URDsMvm8tZuK2ynXv4JwS8UapWGSRHjhBCtBMJ5Ig2tWhbBcF9ozi7fb6ykAqXr/7nJJuBaVcMZ3BWbP0ylQrOHJjG1JN7YTbIgDIhOrriGg+Vex3Xe1tXWBO1LS3WxHvXjqJnSsPIHI1axeVHZXP5UV3QaeTPlxCi40iwGvh+bXHU9um/bqO8NnLgusOTHDlCCNHhyJ2wiCoQDFHq9NZfeCRYDaTYDGj/xA1UtJs2AF8w1CTIkxVvZsaVw6mo9eHyBogx60m06rEZI1eFEELsny8QpNTppaLWh0atIsGqJ8VmRK1u+UScLk+g2XZfIBS1rVuSlfeuOYoKlxe3P0icWU+i1YBFgrhCiA4mEAo1e43jcPujPsjq8AK+yCNydCbwy4gcIYRoD3I1LCKq8wWYv7mcv328ihq3ciNmN2n51/kDObZH0iGPhjmqW0LUtp4p1og3aPEWA/EWmUYlREtwuP18vaqQR79ZXz/dKclq4H+XDmFol1j0Gk2Lvl9WvBmVSqk4ty+bQUuMqfmgbKLNQKJNjn8hRMemVakZ1zeFXzaXR2w/rkcSdmMnvewOekEdZWqV3w2hEKhllKQQQrQlOeuKiHZW1HH9O8vqgzgANe4AN767nB0Vhz6MNivOzIicuIhtD57ZT/LeCNHK1u5ycM/naxvlrCmr9XLFa0sorIpcKe7PSLDomTQqO2Lb3yf0IjlKHhwhhOhMCh1u0mKMEXN7mXQabjg+F5O+kwZyoubIMQFhCLjbvEtCCHGkk0COaMLjD/LKL1sjPkEPh+Hln7fh9jc/XSKaRJuB5y8dys3H52LbPfqmf4adD64/qlEuHCFEy6uu8/H0jxsjtvmCIT5dvotwpAP/T7CZdNw+ricPnNmXpN0ja7okmHnh0iGcNThdct0IIQ4LX60q5IEv1vHk+QM4c2AaOo0KlQrG9khk+uThxJo76ZTwUAhCgcg5cnS7g1aSJ0cIIdrcYXEF/cILL5CTk4PRaGTUqFEsWbIk6rqffvopw4cPJzY2FovFwuDBg3n77bfbsLcdX50vyKbi2qjtm0qcuH3BqO37k2I3cvv4nvzw12OZ/48TeHPKSEZ1TZDkxUK0Mo8/yPay6Bfca3Y58AdbPodDotXA5NE5fH3LGOb/4wQ+umE0pw9MJ9Yc4cZACCE6mUAoxOoCB4UOD9e9vQyDTsN/LhrMC5cOpXeqnTs/Xo23mXxgHVpwd4LmiFOrTMpXX/RrRiGEEK2j0985f/DBB0ydOpWXX36ZUaNG8eyzzzJhwgQ2btxIcnJyk/Xj4+O555576N27N3q9nq+//pqrrrqK5ORkJkyY0A570PGY9Rp6pVr5o6gmYnuvFBsm/Z/Lo6HTqEmLNf2pbQghDo5Rp6FbkpVlO6sitg/IjEGnafmExwBqtYqUGJlGJYQ4/GjVagZlxfLDHyV4AyE+XlbAx8sK6tsHZsZg1LZs/rE2E9gdyNFEuGWQETlCCNFuOn0g55lnnuHaa6/lqquuAuDll1/mm2++YcaMGdx1111N1j/++OMb/Xzbbbfx5ptvMn/+fAnk7GbUabju2Fw+X1nYZHqVSgXXH5eLSdcxf3X8wSAlNV48/iBGnYYkmwFDZ714EqKFxZr13DG+J5dOX9ykTa9Rc+6QDFSq1gnktIfqOh9VdT5CIbCbdPVTu4QQoqWdMSCN5+ZsJjXGyBWju5ARqyR631DsZFROPHGWTjoCMehXvjY7IkcCOUII0dY65t34AfL5fCxbtoy77767fplarWbcuHEsXLhwv68Ph8PMnTuXjRs38uSTT0Zcx+v14vV663+uqYk8SuVw0yXBzKuXD+fvH6+iuk75Ix5j0vGvCwaSk2hu595FVub08s6inUz/dRsuXxCjTs2lI7O54bhcSah6GDhSj8WW1i/DzuPnDeDRr//AtXuKZLLNwPOXDiEj7vA4TsLhMJtLa7n70zX1o49ykyw8du4ABmfFYtRJcPfPkGNRiKYy4kx8euNo8qs8PP3DRjaXKtONhmTFMr5PMqFQGLW6ZQPlbXIs7plaFbH8+J4ROTK1Sggh2lqnDuSUl5cTDAZJSUlptDwlJYUNGzZEfZ3D4SAjIwOv14tGo+HFF19k/PjxEdd9/PHHeeihh1q0352BWa/lhF5JfHvrWCpcPsLhMIlWA8k2A9oOmJzU7Q/wys9bmT5/e/0yjz/EjAU7KK/18eg5/bHvp8yx6NiO1GOxpcWY9Jw/NIOxPRKprPWh0ahIsOhJsRsPm9E4BVVuLnj5t0ZV97aWuZg0fTFf3TKGvun2duxd5yfHohBNaTVqdFoNt7y3nECoYTjzivxqLnplEd/eNobseEuLvmebHIuBA8mRIyNyhBCirXW8O/I2YLPZWLlyJUuXLuWxxx5j6tSpzJs3L+K6d999Nw6Ho/5ffn5+23a2HWk1atJjTQzIiGFgZizpsaZWCeIEgiF2VbtZXVDN6oJqCqvdBIIHlxSwrMbLmwt3RGz7clUh5bXeiG2i8ziSj8WWptdqyIwzMzArln7pMaTGmFo9iFNR62VTiZMVeVVsL3fh9Phb7b2+X1fcKIizRzAU5tnZm6j1tt57HwnkWBSiqTpfgBd+2tIoiLNHrTfA5ysKCUZo+zPa5FgM+pSvEcuP756uKoEcIYRoc516RE5iYiIajYaSkpJGy0tKSkhNTY36OrVaTffu3QEYPHgw69ev5/HHH2+SPwfAYDBgMEhehdbi8gb4eVMZd32ymhqPcuMVY9Lx1IWDOKZ7Amb9gf2KOtz+ZqvtlDm9dEuytkifRfuQY7Hzyqus45b3lrO6wAEoubbOHJjGPaf3JaWFpz16/EHmby6P2r48r4paTxCrQUboHSo5FoVoqtYTiJpIHmDBlnKmjMlp0XNPmxyLgWamVml0ykgdCeQIIUSb69QjcvR6PcOGDWPOnDn1y0KhEHPmzGH06NEHvJ1QKNRojrFoO9vLXdz07vL6IA4oQZnr3v6dvIq6A97O/qpo2YydOmYpRKdV6vRw9RtL64M4AOEwfLmqiGd/3ESdr+nImT9Dp1aRGRe9Il6K3Yhee3hMIRNCdBx6rZpkW/TAdHqcCb2mE+bn2jMiRx3lOkpnkhw5QgjRDjp1IAdg6tSpTJs2jTfffJP169dz44034nK56qtYXXHFFY2SIT/++OP8+OOPbNu2jfXr1/P000/z9ttvc9lll7XXLhyx6nwBXpq3NWJbOAzTft2ONxA8oG3FWwwM7RIbsS0nwUyiVZ4eC9EeShze+qSf+/p4eUGLT3vUaNRMOqpL1Pabju9OvEXOB0KIlqVWwaRR2VHbLxiaiU7TCYPIzY3Igd2BHBmRI4QQba3TD1O4+OKLKSsr4/7776e4uJjBgwcza9as+gTIeXl5qNUN8SqXy8VNN91EQUEBJpOJ3r17884773DxxRe31y4csdy+IJtKnFHbN5U4cXuDB1Q+PN6i59mLhzB5xhK2lzdcUKTajUyfPEKqVgnRTgod7qht/mAYl/fAgrUHIyvezL8uGMj/fbqmUb6KyaO7cFS3+BZ/PyGEqPMFcbj9XDYqm3cW59Uv16hV3D6uBxuLaxieE3dA1zQdSv2InCiBHK1RAjlCCNEOOn0gB+CWW27hlltuidi2bxLjRx99lEcffbQNeiX2x6TX0CvVFvVpfa9UGybDgV/wZMebmXndUeRX1rGtzEV2vJmcRDOpMdGnWQghWld6TPQgqk6jwnIQx/iBshq0nDkwjaO6xrNmlwNvIMSgzFiSbHrsJn2Lv58QQhh1Gr5dW0TfNDuvXzmCzaW16DQqchIsfLq8gJxEC/oOWPVzv5pLdgwSyBFCiHZyWARyROdk1mu54bhcvllTRHifPMVqFVwztutBP7lKsRtJsRsZniNP3YXoCFJjjPRMsbKppGnA9sJhmSTZWmeak0mvJTtBS3ZCy5b7FUKISGLNeu4Y34uJ0xbxzqKdZMSZCIVgV7UbvUbN938d2+rVAVvFfqdWSSBHCCHaQyd8NCAOJ90SLbw0aSgxpoYLhFizjlevGE6XeHM79kwI0RKSbEZemzyCwVmx9ctUKjh7cDq3j+uJSSfPE4QQh4d+6XaeOG8AJp2G/Eo3u6rdJNsMvH31SDKaScLeoe0v2bHGIMmOhRCiHcgVtGgXgWCIslovoTCMyInn+9uPpdTpQaVSkWjRk2w3olF3widXQoh6To+fGk8ArVrFjMnDqazzUesNEGvSk2DVYzNKCXAhxOHDbtJx3tAMxnRPxO0PolKpiDFpSbQaOudoHGgYkRMtR47OBN7o+Q6FEEK0DgnkiDZX4vDw7pI83vhtOzXuAIMyY7jvjL70Tbdj1suvpBCdXSAYYmuZi8e/Xc/Pm8vQa9RcMDSTG0/IpXuyrb27J4QQrcLtD7KxqIZHvlnPsp1V2AxaLh/dhStGd+m8+fqCXlBpQB1lqrvODDW72rZPQggh2m9q1datW7n33nuZOHEipaWlAHz33XesW7euvbok2kC508vtH6zguTmbqXEHAFhV4ODCVxayusDRzr0TQrSEHRV1nP3CfOZtKiMcBm8gxLtL8pg4bRGF1dGrWAkhRGe2vrCG8176jWU7qwBwegO8OG8rN7yznDKnp517d4gCvuj5cUBG5AghRDtpl0DOzz//zIABA1i8eDGffvoptbXK3NpVq1bxwAMPtEeXRBspqKpj4bbKJsvDYXjgi3WU13rboVdCiJZS5w3w3JxNePyhJm35lW5+39H0+BdCiM6u0uXjwa/WEQo3bVuZX8328k6aEDjo3X8gR3LkCCFEm2uXQM5dd93Fo48+yo8//ohe31AK9sQTT2TRokXt0SXRRhZuq4jatrHEicsbOKjt1XkD5FXWsXh7BavyqymqdhOKdBUlhGgTNR4/v2wuj9r+1eoiAsGmQZ79KXN6WV9Uw6KtFWwtraW6zvdnuimEEC3K5Q00O7L4541lbdibFhT0R8+PA8rUKgnkCCFEm2uXhCRr1qzhvffea7I8OTmZ8vLoNwCi84u36KO26TSqg0pwXOny8sZvO3nxpy0EdgdvEix6Xr58GIMzY9FppSibEG1NrVZhM2qprvNHbI+36FEfZNLPHeUurnv790YlzE/qncxj5w4gNcb4p/orhBAtQaNWYdSpI45GBIi3Rr/+6dACBzAix++GYAA0kudQCCHaSrvc6cbGxlJUVNRk+YoVK8jIyGiHHonWVljt5vt1xfRMsREtVnPWoHQSDuJC59fN5Tw3Z3N9EAegwuXjsumLKXJ00rnoQnRyMUYtk0fnRG2fNCob9UEEbEudHqa8sbRREAdgzoZSnpy1/qBH8QkhRGuINes4b0hmxDaVCo7tkdTGPWoh+51aZVa+yqgcIYRoU+0SOr/kkkv4xz/+wUcffYRKpSIUCrFgwQL+9re/ccUVV7RHl0Qr2lpayyWvLqKs1svJfVN44Mx+PPjVOsJ7zYDqlmhh6sm9MOmi/0rWOqrQesoJuR1oTTFUVboxaNV4A42ffnkDIWavL2HKmK6ttUtCiAg8/iA/byonxW5kVNd4Fm9vnA/n1hO7kx1vPqhtFjs8bNudW2JYdgx/H20nTV8HqNjh1lBV58NikKfAQoj2VV3n56oxOaSaQ4zvosYSdBLUmvh5VxhLTBK+Q5hS2iEEfBGnVq0rD/LUEg9P9rKSDErCY1NsW/dOCCGOWO1y9fvPf/6Tm2++maysLILBIH379iUYDHLppZdy7733tkeXRCupqPVy68wVlO1OYvzDHyVo1CpmTB7Bsp2V1HqDHNcriT6p9manSHirdmH44f/QbfhCyYysUjGpxxkMvfQeLnp/R5OhzOsKpQKWEG2ttMbDze8uR6NWcfdpfZg0KpvF2ysx6jSM65NMzxQbseaDm15QUqOMrrtkcAJ/715Iwo9XQJ2Sa6uLPQNf3Ms49cOwWSwtvj9CCHGgAqEQnqpibgrPRPvxNCW3DJCTdRRl45+jLGxv5x4eoggjckLhMHf/4mZ1WYg7fem8HgaVjMgRQog21S6BHL1ez7Rp07jvvvtYu3YttbW1DBkyhB49erRHd0QrqnT5WFdY02jZd2uLmbWumMGZsTx10SByk6zNbqOuphL9rDvRbvy6YWE4jG7TV/QJevnrMXfx+LziRq8Z1iWuxfZBCHFgFm2vJBAKEwiFefDLdcSadfRPj8EfDDFzST4//PXYg95meqwJu0nLbYMh4YMpNBrKV7ML/XvnEbxmAVh6tdyOCCHEQdITpOv299EufrHRclX+IpK/nETo3I+BmPbp3J8R8IG68e3C/IIgq8tCnNJVy6ztRjbpM+nllUCOEEK0pXbNBpudnc1pp53GRRddJEGcDsLjD7Krqo7t5S5Ka/58nhlfIPJQ4nAYVuRXU3wAuWy07nK0m76J2Kbb+gMnd9U0WmYzaBnbWeeiCxFBicPD9nIXBVV1UY+pjqBi98i7Parr/MzfUs7i7ZW4fAECoYPve4rNyO3HppO47NnGQZw9gn5Uy16n1i15sYQQ7ScmWIF1+SuRG8s3Y/UUdejzd1RBb5OpVWvKglh0cElvHTp1mPmh/uCtibIBIYQQraFdRuRMmTKl2fYZM2a0UU/E3oocbp6fu4WPlxXgDYTIjjdz7+l9OKpbAnZTM4numhFj1mHRa3D5gk3aVCrIiDXtfyMeR+QbuN2MAWf9910TLTx/6ZAD264QHVx1nY9fNpfz5Hcb2FXtxqTTcMnILG44LpcUe8er1jSya0LUtu7JViz6g/+Tk2gzcNHAeHSr10ddx1i6klpfHZg63mcihDgyaALuZhP+6qq24sscjr6zVdQMeJtUo1pbHqSLXY1Bq6J3nIr5VQO4WqZWCSFEm2qXQE5VVVWjn/1+P2vXrqW6upoTTzyxPbrUqVS5fBQ63Hy3pphQOMwp/VPJjDURbzUc8jbLnB5ufGc5K/Or65flVdZx3dvLeOXyYUzol3pI202xG/jr+J48+k3Tm7BJI7MPqEqVymhrtj0xIZEvbx6ATqsm0aonySY3c6JjcnkDlNR4+PGPEspqvRzfK5meyVaSIwRlQqEwczeUMvXDVfXL3P4gry/YwaZiJ89dOoQEy6Ef860hO97M8C5x/L6zqknbg2f2JdF2aP01m62E47qiqtoRsT2Q0IuwVo57IUT7KKnxoA3qSNAalMBHBOHYLs0WdOiwgv4mI3LWVQTpl6CMhu6XqOWLij743U4O7ZGfEEKIQ9Euf1E+++yzJstCoRA33ngjubm57dCjzqOi1sszP27i3cV59ctenLeVswelc98Zh36jlF/lbhTE2dsjX//B4KzYQxoBoNNoOH9oJolWA0/9sJGCKjcJFj03Hp/LOUMysBl1hMNhSmq8VNb5UAHxFn2j9wqbEwnljEW949cm2w9nHcVml5EHv1+H1ahlyjFd6Z+hIvFPBLWEaA113gCz1hbzt49X1Q8wm/7rdvqk2ZgxeQRp+4wiK3F6ePzbDRG3tWBrBcUOT4cL5CTZDDx/6VBmzN/Gu4vzcPmC9Emzcd8ZfemXaie/sg6H249ZryHeoj/gxMdqUwyBsXei3fZT00aVmqr+V/LX91dz0YhsRnWN75CjlYQQh69NJU4WbnTxtyGTUS99tekK9gyIy0GjVrV95/6sQONkx05fmLyaMKd1U0YW9UrQ4MbIlvIq+rRXH4UQ4gjUYR4NqNVqpk6dyvHHH8+dd97Z3t3psP4oqmkUxNnji1WFnDYw7ZBHziyP8AR9j4IqN3URpkbtKxAMUe7yEQ6HiTHqMO8uCRxn0XPOkAyOzk3AGwih06hIthlRq1V4/EGW7qjkbx+toqRGeYqVHmPk6YsGMTQ7DoNOg96WiP+sF+Gza1HnL6p/v3DGcLYf9yyXvL2JGk8AgF82lXPu4HTuPaMvCRLMER1IidPTKIizx/oiJ6/8spX/O61voyH3Lm+gvtpbJH8U1tAvvWMkzqx0efEGQhg0alJjjPxtQi+uOqYrgVAYs15DKAwv/LyVNxbsqC/Be0xuAk9eMJDMuAMrR65O6UvgtGfQ/vB/ENidD8dgp3z8szy5yMP8LZXM31LBsC5xvDhpqARzhBBtZsGWcnqkxePtehs6ZwnaDV80NCbk4rvwPZy6JDrlWSnohb1GPG6sVK4Hc+zK36vs3V//qAhKIEcIIdpQhwnkAGzdupVAINDe3eiwXN4A0+dvj9r+6i/bOKpbAjGHkM+muZsevUaNTtP8U6Qih5v3F+fx3pI83L4gJ/RO5q/jepKTaKl/AhVp+sjOijqufH0pwVDD3W2hw8Plry1h1u1j6Z6sTKvSxWcTuOgdQq5ywq5SVJZEVlbqmfLutvogzh6frSzkiqNzJJAjOpQ560ujpnqauTSf647NJX2vUTl6rRq1CkJRXnOoo+9aUnWdj6U7qnjmx41sK3PRPdnK307uxZDs2PoRRr5AkOfnbuHVX7Y1eu2CrRVc8+bvvHX1SJIPYDqk2hyLesgkArkngbMIbxA21Vl56jcHC7ZX1q+3bGcVi7dXctag9JbdWSGEiKJbopWRXeO5Z/ZmusfcxmkT78DgKSOot7PWoeebn+qYOj5ApyzDEPCCoWGKe4FT+aOUYlGu7cw6FanqatZVqjm/XToohBBHpnYJ5EydOrXRz+FwmKKiIr755hsmT57cHl3qFAKhMDV1/qjtDrefQPDQKiIMzorFoFXjjVBR4dwh6SRYok+BKHa4mfLGUtYXNSQd/np1EXM3lPLVLWPITY5cXtzjD/LKz1sbBXH2CITCzJi/gwfP6lc/SkFrSwJbEtCHIoebK2b+HHWk0KfLCxiSfRAlyGvLIORXnjqZ4w/8dUIcoEqXL2qbxx9qchzEW/Sc0j+Vb9cUN1nfotfQI8px1VY8/iAfLytolP9qXWENV72xlIfO6sfEkVnotRpKnd6oAegNxU6Kqj0HFMgBQGdEm5BDpTGNyTOWsGZX09GJAO8s2skJvZKwGSMEtb214HcDYbAmH9j7CiFEM0Z0jWNbmYvPV+yiS4KFdHsivdJyya/288Svu9hRUcf1x3bS1AGBxlWrilwhLDowahse8HXRVrOupvl8hkIIIVpWuwRyVqxY0ehntVpNUlISTz/99H4rWh3JrAYt4/umsCJKLptxfZKxmw7tvzQlxsCMK0cw5Y2ljYI5AzLs3D6uJ6Zmqs2s2VXTKIizR50vyHNzN/P4uQPqp1k1bg+wttDRzHYdlDk9pMWYUO87rzwMoX2GN6hVkGAx4A0EaRTPqqsCRz6s+Qh8ddD/XEjsAdYUcJXDtnnw85PgKICU/jDuQUgdAEZ71L4JcbCO65nEi/O2RmwblBmDZZ9jxGrQcc9pfdhSWsumkoZqICadhjeuGklqO08dKnN6+ff3GyO2PfHdBk7qk0xmnJk6X7DZqZnbympJsxsJqyDRajigHBLhsBLsjSa0b1s4DFU7lGO9YAnE5kDWSCheBWlDwJK43/cUQohoFm6tINas5+EJmVzUHfQVa2H1L/QxxXLSxPNZWJZMVV30YH6HFvQ1ypFTVBsiwdT4PN1FX8N3rnTC4TAqVSfMAySEEJ1QuwRyfvopQsJKsV8atYozB6Xz2vztVOzzdN9u1HLpyGx0Gs0hbVuv0TAiJ47ZU49jRV4VxTUehmbH0SXB3GwVqFAozBcrd0Vtn7u+FMep/oiBHKNOQ5d4S6Ob1L1lxpmY9ss20mJNnDs0o9FT+1iLjjMGpvPxsgLUKvjrmBTO6q7DUL2JsN6GPcMKAT/4nLDgWVjw34YN/z4duoyB816Bxa/Cb3u1FSyBN06DC9+APmeDupOVCRUdVtdECwMz7KzeVdNouVoFD5zZj/gIo94y4sy8c/Uotle4WJlXTUaciUGZsaTFGNFq2vd3s7zWG3EEHyjVtSpqfWTGmTHpNOg0KvzByIEXk17DSc/MQ6fVMOWYHC4anhVxGube4i16LhiWxSNf/9GkrUeyhZfOSsFWshRcZZDUF4IeeOMM8FQ3rKg1wnmvwqqZMHQy7Kc6nhBCRFLr9fPlqkLundCVCfEhNF/eDCXr6ts1C1/gmHEPUZ09sR17+ScEGyc7LqwNE2/cJ5BjrMNZq6fI4Wk0RVgIIUTr6VA5csT+ZcWb+eTGo3nmx018u6aIUDjMyX1T+fspvciKP7CkodHotRqy4s0HtR21WoXdGP3XyKTXoI7ydMas13LTCbn8uL4kYvu5QzL46wcrcfmUhMj/umAg8bur9Jh0Wm45sTtzN5Ty5IRUxuS9hOn9d/faGStMfB+0psZBnD12zof1X0Lewsgd//bvkDkSYjKi7psQByPZbuSVK4bzxoIdvLNoJy5fkCHZsdx3el/6pEUPIiTbjSTbjYzqmtCGvd0/3X4CSdrdebUSbXouGJrJ+0vzm6yTHmPE5Q3i9AbBG+SpHzaxcGsl/504uNnKcyqVitP6p/Luop1sK3fVL++dYuGdMywkvjdBCeIAnPI4LHujcRAHlITJX/9Vaa8rk0COEOKQaNUqLAYtWQYvmt/fahTE2UM1+wFiu48DUtq+g3/WvlOrakOkWhqf/zNNysPFzaW1EsgRQog20maBnCFDhhzwcMvly5e3cm86t5xEC0+cN4C7Tu1NGIgxabEaDj7BcUu5ZEQ27y1pepMGcNlRXZq9IctNtvLk+QO4/4t19U/3jTo1/zilNz+uL8G1e0rG7PWlFDs89YEcgJwEC7NuHUP8+rfRrn238YZ9tcrNm7qZ8sZLpsHwKcoonH25ysBdJYEc0aLSYkz8bUJPrjw6h1A4jNmgJe4AS3B3NIlWPck2A6XOppW10mOMJO4VdL19fE/Ka32NgrZdEsw8cnZ//u+zNY1eu2BrObuq3PXnjUqXj/JaL4XVbhIsepLtRlLsRtJiTbxzzSi+W1vMR7/no1apeOv8dBLfPUk5dveI7QJlkaeAUVehBHvLNkN8tz/5iQghjkRGnZYrj87BHC6D1R9GX3H9F5Dav+061lL2nVrlCtM3sfH1fKIxjAE/m0ucHNezU6Z0FkKITqfNAjnnnHNOW73VEcFs0EacrtQeshPMXDe2G6/+2rgqTf90OxcNz2w254XdqOOcwRkc3T2RvIo6ymu9mHQa3lucx7xNZY3WXbqjir57lVuuqPVSXVZA8m//ibxxjb7xDd2+vDWga2b0kaZjfL7i8KLTaOorOnVmKXYjL102lEunLW40xcqk0/DCpKGkxBgbrfvvCwdSXuuj2OHGatTxR6GDuz9dw65qd5NtL9hSzqCsWIodbv7+8Wp+3Vxe39YlwcyMySPITbaSHmtiyjE5nDM4HbUK4gp/aXrMh/ZTCTHglgTnQog/pU+aHU1dhfIQKZrmrkc6sqCvfkSOJxCm0hNukiNHrTORoa5kc5Sp8kIIIVpem92pPvDAA231VqKNxZr13HRCLmcOSuODpfnUePycOySTvun2Zsua72HQaciKM6MKw9VvLsXjj5x3Y9+y6j9vKiNH54WawsgbLlwOo26ATbMit3cfD8VrIrelDgBTx5rKIkRHolKpGJQZyw9/PZbv1xWzKt/BkOxYTu6bEnFofaxZT6xZT/dkKwVVdTzw5bqoeXNizTpc3gBPfLehURAHYGdFHZNfX8LHNxxNaowRlUpFwp5Rf9URRgaGQ0rpXG/ThOyoNWCKh9jsg95/IYTYI9FqIOQzQM4Y2DE/8ko9Tm7bTrWUgK/+wVaxSzlnJ+yTIwedkXTK2FQS4TwrhBCiVUgmV9EiYs16BmTG8ui5A/jvJUM4oXfyAQVxGm3DouO4npHLAQ/IsDM6tyGwUub08t85m9lU4VcqTUVSthEyhkeeMqEzw9ip0PcsUO8TzzTGwrmvgFWGBwvRHK1GTZcEC9cdm8vzlw7hmrHdyE6w7DcRc6JVz9TxPYk1N50SqlLB0bmJVNR6+Wp1UcTXF1S5Kav1NG1I7dd02Yq3lWM9kmFXKYEca2qz/RVCiP2pDZvhpPsbTUOqlzaYsK0TTtUOhSDkrx+RU+xSHrTF7RvI0RrIpIQtpU7C4egVBYUQQrScdpk7EgwG+c9//sOHH35IXl4ePl/jCkyVlZXt0S3xJ7h9AcqcXpbnVePyBhieE0eyzUhchEo80VgNOu45vQ/rCh0UVCnTLUZ2jeeG43LZWeHipXlbGZ4Tx9DsOFQoN3MvLoFxJ99L4ueXRNhgMpgT4Iov4bfnYeXbStK+nqcoF1tx3SAmG25aDGs/htL1kDMWeoyXJ/RCHISSGg87K1xsKHbSJd5MjxQbabtHy+zN4fZR7PDy3Zoiarx+HjtnAOW1Xv757Xq8gRAqFfz7gkEk2w3kVdYRjFBiPN6i5/kzUunhWAgrZ4M9A/qcoXyN7QLJfaF0r2pWW+ZAygC44HX4+Uko2wBxOXD0XyD3JLClgbZz5ikSQnQcayvVDNFrMV35Hcx9BHb8CgY7DLsSeozHo7Xx50pStIOQX/m6OzhV6VHOyTGGfUfkmMhQleP0Bil1eg/6QZ4QQoiD1y6BnIceeojp06dzxx13cO+993LPPfewY8cOPv/8c+6///726NJhp9YTwO0PYNG3fi4dlzfA9+uK+fvHqxvdeJ0zOJ17Tu9Lks1AUbWbjSVO1u5ykJtkZUBmDOkxJtT75M/Jjjfz0fWjWVlQzeYSJwMzY7n+7WX1OTje+G0HiVY9H90wmt6pNtYX1RCI70n49GdQzXu8oVJN1kg45Qkw2pVpFSc/DGNug3C4YRkow4UTu8PxdylPnqTcuGgjoVCYqjoliB1v0R9wMviOJq+yjiteW8yOirr6Zck2AzOvO4o6X5BfN5dh1ms5rmcSX6zcxX9mb65fbwY7OKpbPG9NGcmibRWcPjCdtBgjZr0Wi16LUaduNNXSoFUz8+JMen4/CSq2NnRi7sNw7qvQ+3S49EP49g7Y/INyvBvsYEmCbicogdqgT5lSZZNROEKIlpMRoyFc64fSddDvXGUkoLcW8hfhNybg0CZ1vkBOYHcy+92BnAp3GI0KLPsOOtIqgRyALaW1EsgRQog20C6BnHfffZdp06Zx+umn8+CDDzJx4kRyc3MZOHAgixYt4tZbb22Pbh0WatxK1YD//bSFHeUu+qTZueWE7nRNsmDWt85/965qN1M/XNVk+ecrCzmuZxKDsmKZOG0RJTUN1W3sRi3vXXsU/dLtTW5g02JNpMWaGJgZy6n//aVRIlWA8lofT363kb+d3IuZi7dhXfMmquIlcPIjSgUajRaKVsN7F8HkbyC5N2gNYE9vfkckiCPaSGG1m69WFfLxsgLUKhUTR2ZxSv9UUmM6VxLk6jofd368qlEQB+DOU3rx6NfrmbuxFFASIKfGGBsFcfZYtK2SdYU13HpSj0bnghS7kauOzuGlnxuSqF84KIHsVc82DuKAErD57Hr4yzJlKuV508FVriQyNtiVoE2k6Q5CCNECar0B0lTV6N89EzKHw9DJ4K5Wrke0BtQLniHm5CeBznWOJ7h7xLy6IZBj16uaPnjQmUimCq0aNpc4OaZ7Yht3VAghjjztEsgpLi5mwIABAFitVhwOBwBnnHEG9913X3t06bDg9Qf5Zk0Rd3/akMB3R0Uds9YVM/2K4ZzQK7nJCJiW8PGygqhtNW4/t3+wslEQB6DGE2DKG0v54pZjSIty81pQVUeNO3LFmVnrirnu2G48dGIy1nffAo8D8hc3XXHdp5D8fwe+M0K0ssJqN5dOW9Qo+PHgV3/w/tI83rhqZNTjoSOqdPlYtK3xVNgBGTGU1njrgzgAY3skMvuPkn1fXm/Ggu2cMTCN5L2e4uq1aqaM6UYgFOathTvxBkJc0NuE8fNPIm8kHIJtPyuBHKNd+SeEEK0sEAzxx65qBjpXKoGPnb8p//aiUanRjL0LbJ2sQt4+I3IqPSHshgjr6cxoVGHSzWG2lEnlKiGEaAvtMgQhMzOToiIliWVubi4//PADAEuXLsVgiPQXQhyI0lovD365rsnycBju+mQNJTURkoP+SaFQmPzKuqjtGXFmVhc4IraVOr2U7hPg2Vutp3EQR69Rc2r/VK4e05XTB6RRUuPB5fUrZcSjcZUpT+aD+ylBLEQbCIXCfLumqMkIFoCNxbXM36dCU0fn8QebLDtzUDof7RPcNeo01Hj8UbdT4/YTjJAgM8lm4JYTujPjyhG8OGko3RKMEIy+Heoi5FdzlUPFFqjc1nnL/wohOqwSp5dF2yoJRzr/7BEOEW7u3NVRBXdfo+01Isemj/BAUKdMGks3+aUEuRBCtJF2CeSce+65zJkzB4C//OUv3HffffTo0YMrrriCKVOmtEeXDgvF1Z4m05D2KKv11ufjaElqtYoTe0euNLWnvTm13ugBlm6JFvaM3h3XJ5lXLh+G1aBlRV41Rp2ajDgTWqONcM6x0d8gYxi8fir89j+oLY2+nhBtoKrOxyfLo49g+2BpPjXuznOxbzfpMOk0jZZZDVoqXY3PNesKHRzVLYFoju2ZhN2o3Cg43D42Fjv535zN/HvWBjaX1gJh1GqoCRmiV6kDXJnHUOxQEqUT8MGuZfD2ufC/YfDcEJg5CUo3KNFtIYRoAZW1XhZvr0CVNSL6SnFdCeisbdeplhLYfS7X7CeQo1YrCY/1HhmRI4QQbaRNAznPP/881dXVPPHEE/zf/ynTXS6++GJ++eUXbrzxRj7++GOeeOKJtuzSYWV/KV5aK5nq0bkJJNmajqRSqyArztTkRq+hP5AeG30aSaxZx8QR2QzvEseUY3KYt6mUUqeXlflVfLpiF4XVHh78MZ/iUfdEzn+x54avfBPMeRAWvQg+96HsohAtQq1SoW7mONSoVXSmnMfJNgM3HZ/baNn6ohqG58Q1Wra1zEWyzUiXhKapPg1aNbeP64HFoKW6zsf0X7cz4dlfePrHTbwwbysXvLyQl+dtw+UNctn7W6k+/jEifUiB7GPY6k/g5XlbKap2KyNwZpwCxasbVtq5AGacDNU7W+YDEEIc8VQqFSvzHQQtKYR7Toi4TvjEe0DXCRMAB/dJduyJEsgB0JnJ0NdQUeujytXyDw6FEEI01qaBnHvuuYf09HQmTZrE3Llz65ePHj2aqVOncuaZZx7Sdl944QVycnIwGo2MGjWKJUuWRF132rRpjB07lri4OOLi4hg3blyz63cmqXYjFn3koEl6jJF4c+uU2M2IM/Ph9aM5oVdS/f1V92Qr7117FGkxRm49qXvE1104LItEa+Q+Vdf5+GxlIaf0T+H+M/vyyfJdbCqupVuShdevHMGVR+ewbGcVP20s594FAWovm0W467HKDZ7BBkfdpFSt+n6v/DgLXwBX9DwdQrS2OIueS0dFL21/+egu2IydJymvXqvh0lHZ3HdGX2LNSr9nrS3mhuNy0WkaX+w/+OU6Hjt3AJNGZmPQqlGpUCpZ3XwMXeItAOyocPG/uVuavM+vW8opcXiwGrU8+Lueiou/Jpw5Umk0xREecwec8V+CwSAqtQqvx0V4/n8aEnXuzeOANZ8oVeqEEOJPSrDosei1qAIeVCNvgBPuBcvuZL9pg2HiTFRbZmOs64TXH3tG5KiVlJoV7nDT0uN76MxkqpTpZcpISiGEEK2pTZMdFxcX89FHH/H6668zfvx4srOzmTJlCldeeSVZWVmHtM0PPviAqVOn8vLLLzNq1CieffZZJkyYwMaNG0lObjrlZ968eUycOJGjjz4ao9HIk08+ycknn8y6devIyMj4s7vYrpLsBp66cBA3vbe80cwBnUbFMxcPJiWm9Z4GdU208NzEIVS6fARDYWxGXf0onYtHZBFn0fPMD5sodXqJMem4dmxXJo7MjnrTuiKvmp83lqFRqXjwq6X1+7NwWwXvL8nj4xuOZvIMJQA3aVAMmlWvocoYBsOvhoAHNn4HjgLIHAFbZisvDvqUHBlxOa32OQixPyf1TmZmhp01uxrndhqRE8eInE6WCBNIsBq4cnQXTu2fitsfxKBVE2/W8cmNR/PAl+tYkVeNRq1iRNc4suJM3H9WX24b3wMAo1aD3aScAwKhEG8vzIv6Pp8s38UFwzJ4ctZGMpNzuemMlzA7NoO7EtWaT9AueIYh8d2IPeUt1m3fRU7eb1G3xba5MOp6MHTCqQ5CiA4lxW5kxpUjCPkKwZGvJDo+6UHloVLFZtj2E2SNIlyxBdL6tXd3D85eI3JC4TAObxhbtGeCehNplKFRq9hU4mRk187390wIITqTNg3kmEwmrrjiCq644gq2bdvGG2+8wWuvvcZDDz3EuHHjuPrqqznnnHPQ6Q78ifQzzzzDtddey1VXXQXAyy+/zDfffMOMGTO46667mqz/7rvvNvp5+vTpfPLJJ8yZM4crrrjiz+1gO9NrNBzXK4nvbh3L6wu2s7nUxaDMGC47qgtZ8a1fCcdm1EUMzMRbDFw8PIvjeybjDSg3er5gmN+2VlDq9DI0O5asODOJuwM/lS4vT/2wkdtO6sGtM1c0SWfh8YcorHbjcPsZ3iWWoc65mFbOaNqhDV/DxPcbAjlQn5BPiPaSGmNi2hXDWby9kveX5KFSqbjiqC4M7RJHir0TDr0HNBp1k2mSAzNjmTF5BDUePxqVijiLHp1GRXGNl5V5VZTX+hjWJY6MOBOJVgPBYJgKV/Tk5zVuP2a98ifrhEw15s+vhOI1jVeq2ErXuTfjPeEVsCZDdZTAkD0DNK0zQlEIcWRRq1X0TrNCWRi+vn13Bb25jVca9xCkDW2X/v0pgYZkxw5vmGAY7M1MrdL6HKTFGNlc4my7PgohxBGqXcqPA3Tr1o2HH36Yhx56iNmzZ/PGG29w5ZVXYrFYKC09sKS0Pp+PZcuWcffdd9cvU6vVjBs3joULFx7QNurq6vD7/cTHR35y4PV68Xobbi5qapqpkNSKXN4AFS4fvkAQi0FLis0YMZGwWa+ld5qdR84ZgMcfxKTXoNM0nkHnDwaprvOjVqmIt+gpqfHiDQTRadQk2wxoNS0/406lUpEaY8QXCLJ4eyXXvPl7o8TMQ7NjeXHSMFJjjHgDIUpqPDg9ATz+yNMflu6oYnS3eCYPMBE7/4XIbxoKKCXJM4YpSU8zhoI5scX3TbSNjnIsHqxQKEyJ04PLG0Cv1ZBo1ZMaY+LswRmc1CcZFWAxNB+89gVCVNf50GpUxFuUgKfLG6DK5SMYDmMzauuXdyRxFj1xFiVg4vUHmb+lghveXoYv2HBcj8yJ53+XDiHFbuS0/mnM21gWcVtHdYtn7S6lAl6arrZpEGePopXYcFM66GaSC66KvM6oG0ArgZxD1VmPRSFai1ajIfjHF0oQJ5Jlb6Dqc3aLv2+rH4vBhmTHFW7lqZq9malVuPLJiDWxUSpXCSFEq2u3QM4eKpUKrVaLSqUiHA7j9x94xZby8nKCwSApKSmNlqekpLBhw4YD2sY//vEP0tPTGTduXMT2xx9/nIceeuiA+9QaCqvdPPHdBr5ZU0QwFCbRqudvE3pxSr9UYqPkvdFr1ei1TQMyeZV1vLNoJ9+tLeK6sd1QoeJ/P22mpMaL3ahlypiuTBqVTZKtdUYGFNd4ufqN3xvdyAEsz6vmpXlbuOf0vhi0GrolWgiEoleW+XhZPq9NHoHKuQucxdHfsKYIzAkQ2wXOnwGW6JVzRMfWEY7Fg1Vd52PWumL+PWsjFS4fWrWK0wakcdepvUmPNWHdTwAnFAqTV1nHjAXbmbuhFJtRyy3Hd6dvhp2nvt/ErHXFBENh+mfYefis/vRNt2OMkly8vZXUeLjurd+bHNdLdlTy2vzt/H1CT8b0SCQ9xkihw9NoHaNOzYXDs7ju7d+VBX5Xs++VrPPgSutPaMT1qJe+0tCgUsHJj0F8txbZpyNVZzwWhWht6sqm+b3qOQsJomnxi+5WPxb3GpFT5VHO3c0lO8ZbQ0a2KWpAXgghRMtpl/LjAPn5+Tz88MN069aN8ePHU1hYyLRp0ygqKmqzPjzxxBPMnDmTzz77DKMxcuDi7rvvxuFw1P/Lz89vs/4BlDk9XPvW73y5qpDMOBMDM2PwB8Pc9ckafvyjhPBBlNHNq3BxzgsLePWXbfRItlFW6+XeL9ZSUqP8oa7xBHh29mYe+2Y9jlYqgbxsZ2WTIM4eH/yeT5nTQ7xFz40ndCfRqkcbpXy5Uach1qwlPTEWUgdGf8Mux8Bx/4Ap30N81xbYA9Fe2vtYPFihUJgf/yjhrk/WULG7gkcgFObLVYVc99bvlDmjTyPaY3uFizP/N5+3Fu6koMrN+iInao2KSdMW1wd2AdbuquHCVxayrQOXfV2wtSJqcPbdRTvJq3CTZDUw8/rRnDskHZ1GqeA1pnsCL00axrOzN9WP0DPZkyJWrgJApUbndxA7fQTq2Ey4YQGc+Ryc8xL8ZTkMvQJMsa20l0eGznYsCtHaQqEw/pzjo7aH0wbjV7f8A7JWPxaDu68FNVqqvcr52xotkKO3gtdJdpyRSpeP8tr9/40TQghx6Np0RI7P5+PTTz9lxowZzJ07l7S0NCZPnsyUKVPo1u3gn5AmJiai0WgoKWlcCaCkpITU1NRmX/vUU0/xxBNPMHv2bAYOjB4IMBgMGAztN2Uhv8qN1aBl+uTh7KxwUeb0cvWYrrj9QV76eStjeiSSFrP//DfeQJBXft5G5e4byvOHZnLnx6sirvv5ykL+clIPYkwtXz2nsNoTtc3jD6FRA85ixiQHKPPpuePknjw5a2Oj9dQqePjsfryxYAejcxM55aT7Ub17QdMN2lKhy9ESwDlMtPexuD/hcJjCag/rCh1sLauld6oNi0EbcYTJ2sIaCqrq6hOCR+LyBnjq+404vYH6ZblJVgqr3U22BxAMhfn39xt5buKQjlP5KhgAVxkQJsUYxKTT4PYHm6zm8gXZXOrEFwzRJ83OPaf1ZeLIbCpqfRh0aiDM6G4JnDkone7JVhaVVnFinwsx/PFh0/cccAFs/lH5/sf7YN4TcNNCiOvSqrt6JOnox6IQbc0XCEL2MUq1Kld50xXG/o2AquWnc7b6sRhsGJFT7VXO3dZof14MNgiHyLYoAfeNxU4Su8t5QgghWkubBnJSU1Opq6vjjDPO4KuvvmLChAmo1Yc+KEiv1zNs2DDmzJnDOeecA0AoFGLOnDnccsstUV/3r3/9i8cee4zvv/+e4cOHH/L7t4Vih5tJo7K5+d3ljXLK9Eqxcd/pfXH7mt4URVLl8vPt2obRTiqVcvMUTX5lHblJLV/RZUROXMTlRp2ajy7tSvLqV2HJy+g8DtK6ncCko++i35VDeP7nneyqctM7zcbEkdn8sK6YUd0SKC0tQqVdAWc8Cz8/Cc7d+9jlGBh7B1RulUCOaHXhcJg/imqYOG0RNe6GwEtajJEnzhvA1A9X1Y/K2WPNLgdDsiMfD6Ak9/3xj8ZB6j5pNn7fURX1NYu3V1JZ6+sYgRzHLvh9Bix/A/wejut5Kj9MvpUbv61ibWHjkUPdk60UVLl5d3EeL1w6lESbAbNeQ1mtl6o6P2admp6jbPzjkzU8/PUfaFQq3rr4doaY4jCtelOpVKc1wuBJkD0KPr+xYeN+Fyx4Dk75J2jlpkII0fL8wRCEwXjuK/Dzv5T8fAAxmXDcXaj++BxLXC4Q257dPHgBL6g0oNbg8AYwaog6UnpPFcAUrQuDVs36ohqO6S55CYUQorW0aSDn3nvv5fLLLycpKal+2fvvv89ZZ52FxWI5pG1OnTqVyZMnM3z4cEaOHMmzzz6Ly+Wqr2J1xRVXkJGRweOPPw7Ak08+yf333897771HTk4OxcVKfhWr1YrV2vFK0XZNtHLuiwsaBXEANpY4+XhZAfee3vuAtqNS0SiJsd3Y/H+91dDyvxoOt48+MQG+ntyVbZVeXllWy7pCJTHff05Pp99vt6POb0hSrdr4DfYtP9Lrku84OjcBi0HLznIXf525Eqc3wPG9kumarIdP/wnpQ+DEe8BgB7UGdi2HT6+BPudA98j5j4RoKSW7cz/tHcQBKHJ4eObHTVx5TA5P/7CpUdu+VZ4i0ahVjaYj1XoCZMZFf128Rc/vOysx6TUkt0YFrNoSZZSN1tgk31QoFKa81ksoDEnhCjTvng9l6xv2Ze1HZG2exUsXfse4N+oandNuPD6XF+ZuYVu5C6c3gFmvjNyJMenokqD8bfjfnM3M36I86Q6Ew1z+wU7OG3ABl59zGdmWALGBClj1Pnx2Q9OEozt/BW8NaJMQQoiW5g2EiHGXwGfXw7Cr4JjbIBwGdxUsnQZFq1APm9Le3Tx4QR9olAcDDm84+rQqUEbkAGpPFVlxJjYUS+UqIYRoTW0ayJk6dWqTZddffz2jRo06pKlVABdffDFlZWXcf//9FBcXM3jwYGbNmlWfADkvL6/RqJ+XXnoJn8/HBRc0norzwAMP8OCDDx5SH1rTtrLaqJWbvl9XzJ2n9Dqg7cRb9Fw0LIsX5inJ+CwGLcO6xLFsZ9On+4lWPfGWQx8CHA6HUe2Vv8Lp8VNVVYGtYg0xvz1G/4ot9O96HMceezG/1OXw8tJqjk9wNAri1Av6SFzwMPqUB3hsduOkxlq1iuq6kJLMeNdy5d++4rL33+GaQnAUKKN54rqCLQ2scsMnDlyp00NxTeRpg6sKHNxyYo9Gy6wGLX1Sbc1uM86i57whGby/tCHnwW9bK3jpsqG8szhyWe0Lh2fx7uI8MuPMLRvIcZXB5tnwy79AZ4LUQTBkEqQOAGMMJTUePl+xi7cW7iTeouPdY0qx7xXEqed1krrqBS4efDVv/V5KlwQzN5/QnSXbK9lW7qJHshV/IMRzczbz7dpiLHoNU47pyujcBDyBIGoVHNczmRN7J6NRw6JtlVw0cxeXDEniAf/rqDZ8E7n/lhTQ7Gc0Tk3R7vNAIcTlgC1dzgNCiAOiUoFKZwS9TRkFrNaCv06Z4j3gQqjYirozjggMeEGjXA9We8LRp1WBkiMHwOMgK74LfxRKNTshhGhN7V616mCS9UZzyy23RJ1KNW/evEY/79ix40+/X1vadzrG3gKhcH2y0/3RadRMOiqbb9YUsqOijvJaL7ed1IO7P13Drmp3/Xo2g5bHzxtImdNDt4OYWlVV5yO/so4Pf8+n1hPgrEHp9MuIIdFqYGVeJf2qfiLu+5uVxMMp/aHgd2Kr13Fmt2SOye0GS5+Pum3Njp8ZPaxxYKlPmo3t5S78AQNjR92KbtsPysVG0Uqoq1RWUqmh95nNd7xsI7xzPjj2ShCYMRwuelMZEi3EAXB6As22+/dK8G0zaHnr6pGk7ie3lVGn4eYTujNvUxlFu3Pi+IIhfvijhIfO6seDX61j79PncT2TyI43szyvmt+2ljOqWwtVaPM4Cf/6H1QVm+CUJ6B0PbgrobaEcIWZansfbnxnOcvzqgEYlRODfdOnUTen2zKLmy/7B6N6ZlLh8vLar9vZWOJEr1HzrwsGcsHLv1Fe60OnUTG+bwo7KlzUeP1cMDSTYV3i+XVzGV+vLuTy0V24dkxXrj+uG4FAiFDoFjTRAjlj/gpGe/R9LN8E71wA1TsblqUPhYvfgpisQ/jQhBBHkkAojNoQA+e9Ch4HeGqUaZ0qtVIl79IPCestNDOepWMKeOtH5FR7w1h0zeyBRqeM1nRXkR3fl/lbyvEHQ+g07VZXRQghDmvtHsgRzRuUGRu1LT3GiPkgpkClx5p4/9qjWLClHKNOw98/Ws3dp/XGFwixubSWjFgTqTFGnp29iUfO7n/A262q8/HiT1uY9uv2+mWfryykf7qdFy8bhtFdSvwv98JpT8PWOfDTP+vXU817grgT78eXMwZ+fynyG+hM7D0oyW7U8vcJvdlRXkufOA3BuOPQustQ+ZwwaCJ4nfDjA3DOC2BPj97xmiJ490IliBPbBUZdp3wNh6F4LRhj6+d8C9Gc9FgTKhVEikubdBp6JFv59wUDSbYZ6J5sJTXGhCZanoG9ZMab+fiGo/lpYynfrCki1qTjvCEZpMcY6ZIwgtUFDty+IIOzY9le5qpPYJ5ka7nROGFXKaqKzdDnLJh5KYQaglaq9CHYzn+D1QWO+mWeQBi/yU7UB7d6C1V1AW5+r/EIustGZ/Ph0nzKa30Mzopl6viefL26iK9WF5FkNZBqVyqhrClw8PQZWehrd2Jf/RWEwzi6nk5lXDoJJ9yL+qdHG7/fMbcpQZmaQmWag1oL5niw7B5t4yyG9y5uHMQBKFwOX0+F86aDKeYQPz0hxJHAqNOgCqghGFTOLQa7EsgxxkAoqJx31J3wkjvgqR+R49hfIAeU6VXuKrpmWfAHw2wuqaVvejNBdCGEEIesTf+qhEIh/v3vf/Pll1/i8/k46aST+Pbbb8nIyGjLbnQqGXEmRuTEsTRCgtN7Tu9D6kFOn0iLNXHB8CxKazykxRq5beZKkqwGMuJMlNd6Kahyk2DRkxpz4NvNr6hrFMTZY21hDZ8uK+DiLl7lwsZfB+u/arKeeu7D6Kf8qAzL9TUtnxwaNInSoI3bTurOMd0TsRt1VLv9HJetJ2n9mxg//VfDysveIJw9mvAN8ykNx2IN67CGw8q0KVe5chNqSQKdGbwOOOFuKFoNXcfC7IegbIOyHXsGnP6MshyUp2tavTKNS4h9JFj0XDA0k4+WFTRpu/mEXLokmOmR0vxUqmgy4kxcdlQXzhuSgUajwqDV4AuEmLFgO79trcSgU/PGbzvqq0Fp1CrG9mi5BJPB8i1oR1zTJIgDQOEKNItf5PR+5/PFGiV/za+bKyi98FIy1rwXcXuO/pNZUqbl1cuHMWdDKTUePxcNSWVUcogtJQ4GJmaSlZ7OtW8tw6BTc9HwLIZmxxEMKUHcaRd0wTjvYUzrZtZv0/L7i7h7nYvzxIew9DkHdcEi1OEQdBkNpjjIXwhf364EbQBS+sG5r0BCD+Xp+UkPKLm16iph4f+gfLOy3pYfoa5cCejuqURjTgBNJ7whE0K0Gr1apQRszLHww/2w9Uclsq+3wFE3Q//zUe17/uwMgj5QK2H5Km+YRNP+AjlWcFfRJcGMCli7yyGBHCGEaCVtejX62GOP8eCDDzJu3DhMJhP//e9/KS0tZezYsW3ZjU4l0WrgfxOH8PLPW5m5NB+PP0RWvIl7T+/L6D8xdSLZbuSZiwYx8dXFFNd4KKtVSkxa9Bpeu3LEQQWIZi6NnK8D4L0leVzVOxX6nw8r342+kWWv47via/SOHeB3wx+fKyWE47tSPOAGehrj2VZVwnVvL6O6zk+q3cj3l8RgX/ivJptS5S3Eu+YLXqo6gW7xJi7LLkfz6RTliTwoT4yO/z+o2g6rP4RJH8PbZ4PP1bCRml0w8xK4eg4sexO2/KAEgMbcDjljwZp8wJ+POPzZTTruPKU3WfFmps/fRo07QJLVwO3jenBq/zT0Ws2ffo+9R9/ptWquPy6X33dWNRoNo1GreP7SIS2bHye+G2yb2zSIs5tqxdtcdsZlfLFG+blnqg2fPZPAiOvRLn2l0brB9KFsTz+T+9/+A6NOzQm9k/nH0TEkbZyO+cvXGOhxMCBzJFWpD3LF8GSO7p3JK79s49VftgEwvk8yJ42qRL9XEGcP08bPqOtxNp+Eh7GheAjXH5urTF8rWArvX9J45ZJ18PppMPlreOPUhmM/NhtOeRzm/wcKfleCQOEwzH8WVrytJFEecCEMuxJiZcqVEELhqPNjVKngk2ugZG1Dg8/VkFtswAXRN9BRBTwNU6s8YXLs+5kmZbCDSxn1nR5rYs0uBxeNkHOlEEK0hjYN5Lz11lu8+OKLXH/99QDMnj2b008/nenTp/+pMuSHu9QYE/93Wl+uPTYXfzCEWdcyFWm6Jlr57OajWV9Uw6r8anKTrAzpEkd6jAn1AUz7AKVSTU0z+UHcviDV2ImJyYS6iqjrqWtL0O74GeY8CAY74cGXEbjuAVaVw9erfGjUBbw2v2HUT/8MO8Y170TdnnH5q5w1/mQS9G4075ytzPPew+uE7++G81+D2lJY/0XjIM4e4TD8/LiSANlZpPz7eAoMuRzGP6xMzxBitySbgZuOz+WCYZn4AiEMOjUpNuMBH0sHKzXGxGtXjmBHuYvftlSQbDdwTPdEku1GTLo/HzjC74HSP9CsfK/+Qj7yenUYNcqcslP6p3Ji72TOfX09fzn6Ik658GySNn+INugi0O8iFrpSuflDJfDr8Yc4o5uWzDm3oC3Yq1pdwRLiZ57B1Mu/5uwv17OhuGGUXvc4FdrFL0btSsLqVxhy0nT+8clOftlUwRfX9Mc65+HIK3trYMNXyuic/CXKsuo8+ORaJUfWuxfCGf9VArp7RugA/PoUrJ4JV82SYI4QAgCtTg0VJY2DOHtb8F9lempns1eOnBrfAUytMsbWVyvMSbSwZpej+fWFEEIcsjYN5OTl5XHaaafV/zxu3DhUKhWFhYVkZkpi2ebotWoyDqBc8cFKizGRFmPixN4pBIIhims8zFlfQqnTw6iuCdS4/Xy5qpBwGM4ekkFOgpkEa0PlBbVaxdmDM/h6dVHE7Z/YJ5mQKZ7qxGHEZh2ljLSJJHsU6vVfKMETjwPVohdQ5y0ifvx0LhyewNnPL2i0ulmnxuCJHhjCU4PNbCRh08eNgzh7WzoNRt0Iy96Ivp3itZB7UuNlK96G0TdLIEc0odWoD6iseEtJthlJthkZ2bVhdJ7D7Wd7uYsvVuzC4fZz+sA0eqXYGgV/nR4/5bU+lu2oJBiG4TlxJFkN2E17BWxK1sGM8aiMMUqS42iSeoPOgkGrTIO65s2lhMLwxUYPtthsug1+gFAoTIJVzyPfLKPWqwR+7UYtQ6zVjYI49cIhDD/ezeTB/+XuWQ2BHLsO1N5mKqF4a/B6leN9a1ktPrcTitdEX794jTLiaE8gB5QpoAVLYeAlSvB27yDOHo4CWP2BkkRZ3QJBMyFEp2bDC6V/RF/BU62cWzqb3YGcUDhMjZfmy4+Dkk+srhLCIbolWvjw93xJeCyEEK2kTQM5gUAAo7HxSBKdToff72/LbogI/IEQy/OquOqNpdT5gjx0Vj+e+mEj368rqV/nrUU7OX1AGg+e1Y8kW0MwZ0BGDH3SbKwvcjbaplmv4dYTe5Adb6G0JhPb2L+j2fRd08CKJQlSBzZKggygKVyG3bWTrW4zgVCYUV1juXmYlXhDGHdYS5X2SuLWfxlxf4LdTmRrhY+eFauj73T5JmX+ekwzOZpiMhpyY+xt+y+Q3Cf664RoB9V1Pl5fsIP/zmkIPsxcms/grFhevmwYqTFGqut8vLc4j3//sLFRcuYbj+vGdcfmEmfRKxfis/6h5Hyoq1SO2ZT+EZ82h095guy0bL6+NZmX520jFIZbTuxOvFnPf2ZvpsjhwWbQcumobF65fBjnvfQbDneA7sk2rLvmKnV7u4+D/hfsrnhSCcvfhsLl9D+m8cX//AIfk3ImYC9c3qQfAL7upzBzTUOgp7AmRHxMppLkOJKYTGWK5b7KN8Oom2DOA9E/7DUfwtDJUqJcCIEqWIfKmhJ9BY0eNJ2w/HjQC2odTh+Eofny4wCGWGUarkcZ5e0NhNhY7KR/hiSMF0KIltamgZxwOMyVV16JwdDwx8zj8XDDDTdgsVjql336afTStaJ1lNR4uPL1pbj9QTLjTOg06kZBnD2+WVPE2YPTOblfav2y1Bgjr185gncX5/H+kjzqfEFO6p3M7eN7kpNgQa2CYBg2+FPoceX36L7/B6r8xaBSE+p5KuoRV8M3d0Tsl3HnHOL6DeK5s7I41vcrsT/9B1xlYEkkcPRf4ZyX4PMblWDQ0MmQOgBCAWoTB/PZj5WMTeqHlR8i73R8LuxaCn3OVPL3RCo5NOxKmPek8r3ODEMug27HgykeqvPBltr8tBMh2lBBlbtREGePlfnVfLwsnxuP786W0lr+9f3GJuu89PM2js5NZGzPJCXpeMHShsYf7oXzpsG6T2HdZ0pgJ7EnwQlPsEnbk7dmbWDyUTlsK3NxTPcETDoND3/d8HTa6Q3wyi/b2Frm4p2rR/HV6iLmby4nYEqEc1+FopXw3Z1K4uGYTBh9C/Q8mUC4cSBnwdYqSo47E/uqacqN0dArILmvkldr8w/U9pnI2s+LefvCLLKMHpKsfhj7N/hoctMPS6WGnhPg/YlN29IGQ2J3OO4fSp+Cflj1HmyZ07COxqBsQwhxxAuHwkqOLUuSco2yr/7ngb7tRmu2mN0jcqo9yvWRZX8jcoy7Aza1ZXRN7IFGrWJFfrUEcoQQohW0aSBn8uSmF9OXXXZZW3ZBRFDm9FDj8XPfGX35ZHkBgzJj+GpVYdT1p8/fzujcBGxGJYARDIWpcPkoc3qYOr4neq2aP4pqWFfoINlqoNDh4aJXFjIwM4ZjeySi7fIEI0ZpiDEbSIwxY3r9JCVXTQQBQxy6kJdxjk8wL362ocFVjvbHewgddTNM/Ag1Ifjtv0r+Cr0Fw8DL+euoK6nSXIj19+eVG7F9jbga5jwCFVvh1H8rN6sBj9Km1sDoW5UEyY58JYHf+dOVaVgfXKYkPTXYYMxU5WbS0nJVgsSRp7TGUz/8/M/kv/ro9/yobW8t3Ml5QzOY/uu2qOu8+PNWBmfHYlOpleScfrfS4HEov/d9z4ZzXyFsS8dnz+aaT/L4dfNKAM4dkkm3JAvH9kzi/i/WRdz+7PUlTByZhUGn5ukLB2LTZ8Ds+2DD1w0rOQpg1l2Exz/CivKm05au+aKU7yf/iMGxFdW8x2HeE2CwExo+BU9IxWtja0mcfU1DharTn4FjboffnlOOW1CCsme/oIz82TeJs9YIvU5TAsSbv1cCvMYYOOomyD4a5j6irDfiGrBIFTshjnQFVXXYtHHoNbWoLv1AqfC35/wDkHsCjL6FEGo63UTMgDIix+HbHcjZX44cU6zyta4cfVIvuiSYWZFXxeVHdWndfgohxBGoTQM5r7/+elu+ndiP6jofS7ZX8uSsDWwtc5FsM3DZUV12lzuvjPq6Wk+AQLBh9MquajcXv7KIWm+AmUsbl19+95pRvPjTFhxuP2qViow4Mze9q5T4Hphh555TchnQ40zMK16L+F6OrHFYAxWYf38hYrt67SeE+58PM05uuCHzuTD+/jK9dv7KpvEzCEz8BO1nUxqmSOlMMPYOJQdIzS5Y8zH0Op3wDQtQOfKVocSJvZQbuNd258c58R5l6lfRyoY39zphzkOg1io3eVKSWBykSpePXzeX8fQPm8irrCMj1sTt43pwUp9k4i0HNww/HA5T6fJFba/1BgiGoLgmSs4ooMzpxRsIYbMkER58Gaql0xoagz5Y8xGs+Yiiy+ezvkjNr5sbph1uKKrhnCEZ1HoCONzRp8sWVLmZuTifYdlxpBpqids7iLMX1a9Pc+qlp7JvquJT+qWgqi1C9d5FDYEZbw3qBc+StnMhqkEXNb6J+mYqDJ8CNy2Cym2gNUFCN9BZlKDR5u8bpntaU+DCN+CXp2DTrIZteBww73E46X7ocgwEfNBjfNR9FEIcGQoq69hQ4mRsih9V2A+/PA0nPdAwTdSeASXrCAc81KpsdLpxKQEPaI04vMo1336nVulMyr7vPgfnJllZkVfdun0UQogjlNx5HuY8/iClTi9rCqpxeYMMyool2WbAZtTy3dpi7v60IRFoqdPLMz9u4qLhmVx/bC63vL8i4jZP6Z+K3djwq/PrpjLc/mDEdZ/6fiNjeyayLK+KY7onMDY1yMIbctlR7SfPZ+Pfs3fw6InX0rNgIeqyxokCneOf4rMtQS7o4ow8okZnhsETUc1+IGJpZHXZOrpTgLbbiXD9L8qFhdcF8V2BEGyZC2PugC5HweoPqQ1q8XQ9maS9R0Rc+zNs+g7ichoHcfb261PQ71ypYCMOitcf5MPf83niuw31y3ZVu/n7x6u55YRcbjqhO2b9gZ+iVSoVZw1K56soiceP75VErEnHMbkJrMyvjrjOsd0TMWrVVHhC2EffhmbHfNS7K5DsUT3uKV5f7WFYjxB90mxcN8xOboyKsNqFSm8hxtj89AG7SUedL8COchfHxkbIT7OHp5oErYfPbzqaFfnVxJp0dE+2kq6vw/DZDQ1BnL0/g4LFcNT1SuUUT7Uy/YowbJsH6UNhwMXgroBwUBl1N/JaZZRRbakyRdKaDLVlsOaDyH1a9CL+iR8TMCVhsqc3u59CiMObNxBkR4WL699exuY7ekPVDtj4DWz5QRm9Z7TDzgVQV4lqy2zM50yHzhbKCXjAYKNmdyDHvL8ROSqVMlLRqYzq7pli48c/Sqh0+Yi36Fu7t0IIcUSRQM5hrM4X4KcNpdz+wUr8u0fQ9E61ceeEXmQnmHn8u/VNXpNsM2DWa+mdauOCoRl8vrKQQKhh9E2iVc95QzPQaNSUOj0UOzyY9BpevmwY28pq+d/cLfVVaQA2lji5bVwP5t40CHvhfKzvPYatagdp5niGj/oLuceexqUf5PPfM16lRziPuILZqG0p+HufxcJiPZaQmoSYvX5N9VYYO1UpGeyugpQBsODZ6B/Chm9wZB5LjCUFfHXgLFHW1xqg56mQORzevQichahMGbyU142bT+jRUJkrLhtGXqeUG47G41ByighxEEqdXp6dvSli2yu/bOPC4VlYDUGq6/yEgVizjkRr86N0BmTG0DPFyqaSht9HnUbFaf1TuW1cT9RqFZeMzOKN33bg8jUOvp47OJ1zhmZwz2dr2VjiZGBmDP84bybasvUYtv+A35QEfc6mKBhLV12AvvHw6Xg3prl3KInDDTaCw6/FPXgKx/dMZN6mpknCY0w6tGoVLl+QkUlB1P6mwZi9FdQE2VDjJNlmZF2hgxpPgB7dQ1C0qpkX/a5Mm8wcDqiUnDvFa5VArM+lVKn65UnlqfHJjylVt+L2Gva/4Zvo23aVk+eES9/awKc3JpAR1wlzXgghWkS508t/Zm8mGAqDwark0Bp5HXQ/CXbMV6amnvaUkjPnx/vRhj3t3eWDF/CCRo/DG0YFmA7krsEUDw7lgUKvFBsAy3ZWMb5vM8mghRBCHDQJ5BzGiqo93PL+ivocvmcMTOOE3snc/+U6/u+0PtS4G49iuePknqTHmPh0RQG3vL+C0bkJfHD9UTz+7XrW7KrhrEHp3HJidzLjzGwvdzHljaVsL3fVv35odhzPTRzCLe8tp273TeJxPZKIM6oxb/kK65w7G96srhLdTw8xeMAGrh5+HZd/mEdWnInnL32Adxbt5NPndygXR0Dm2ZmcmthDyVdzwQyY/x+Ys3vCxfmvKflrPI6In4FbH8/7S/K4foAG1bd/h20/NTQufEEpP37RW/DaONwxucxbWM55Q7MalVjHVQHW1KYb30OtVYYTC3EQKl0+PFECGf5gmCKHh0e+Xse6QqUaXI9kK0+eP5D+mXb0msiZFlJjTLxx1Uje/G0H7y/Jo3eqjb9N6M1Xqwq56d3lpNgN3Hhcd778yxju/Hg1y3Yq1ZwuHJbJib2TOfuFBfXniw3FTj5aVsDTFw5ip7kbJ/RO4e5PVrO+eAf90u1M0JZj+urKhjf3OtEseAbTruX896yXOWOGi/xKd32zWa/hXxcM5Lk5m9GqVeQYasAXAlNcxKpS4ZwxaGzJvPn9lkYV8QZOymaI1tC0+t0ethQwxMB3/4Az/gNvn9O48pzODOe9quTWmX4SXDsX0gY1tMdmR94ugNZIXUjLpEFxmOp2gcqg9N9gjf4aIcRhyR8M159Dw2oN5J4E+YvgvYsbVlo6HXLGwLmvEkbFfsazdDy7c+TU+MKYdaBWHcAemBPqKxwmWvUkWPX8vqNSAjlCCNHCJJBzGPIFQjjcPj5fuav+pizBoufMQenc8M4ywmHQqhv/Mb7qmBxKarw8/UPDCIENxU4+WJrPR9ePJtGmx27UYdJrKa3xcM2bjYM4AMvzqnh74U4uHZXN9F+VKRM3n5BL0FFA7ILGpcX30K75gLMn3cJTv8JRuQlM/3Vbk6khj8wrZ9AF00jf9T0seQXyFkK3E5RpEWqtkmz4t/9F3L4z9wzCeV7YOK9xEGePxS9Bz1Mg51jK44ewrTyPNQUOpcKC16k83f/+biWnTrRqFAMuBEtyxPcXIhqNuvkLYqfHXx/EAdhcWsslry7i29vG0j05euAgPdbE3yb05OoxXSmsdnPxq4vwBpSA0YZiJz9vKmfq+J68fNlQar1KwFWjgrP2CuLsEQ7Dg1+u440pI7nv87WsL1b6c/soGwm/3hR5v3bMw1BXxP8mDqHK5WfhtgrSYoxkxpl4fu4W1hXWMK5vMmFnsRKUPet/8Nn1ymiZPWKyCJ36FC/NL28UxAH4YUeQgQMuQbPizaZvrlJBtxPh99dhwhMw+8HGQRwAfx18fbsyGuez6+GH++CiN5WADChVsPZMzdqHd8ClZMYYuKn8AbTT5ypVq3qdAeMfVEb6CCGOGHqtGrNeQ50viCdsxGxNRvXbc01X3DEfck8i3PW4tu/knxVsGJFj3d+0qj3MCcoDsKAPlUZPzxQbi7dHz7sohBDi0Ejt1MNMaY2Hf32/gUe+Xs/Oirr65ecPy+SNBTvqb9S2lNbSP8MOKDeUY3sk8c6inU22V+cL8uBX6zBoNZh25+sodXrZWuZqsi7AvE2ljOqagFat4vaTehAGbGFXxJuiPZJ8+Vw7pivj+6Ywa11xk/ZCh4cbvncR7HO2MnS59+nQ7xz4+Cp4/xLIHq2UC95HeMITOPXJXDPIjGp5M4m2V75D3enPc/ccpY8J1t3zuPMWKU/zS9fDz/9SqtxY9wnYdDlGSYCqN0ffvhARWA1achIi/96k2o1U1TXNC+ULhnjt1214A5FzUu2h02jQqFX832dr64M4e/vP7E3UeoN0TbTQNdGCw+2nOsL7AdR4AgSCIdbsahj1lmkOQnVe1Pev3bqIdxblkWDVcdmobPIr6rjz49VUu/1MHd+TswamU6VJhOI1MP8ZZYTMKY8rZcfPfh7GPUhd2MAvm8vIjjej0yg3EJlxJqb016Dpfw4k9Wr8pioVnPUCOIugeqdyTBavado5UII7Bjtc8JoyDatwJVRuB2+tkpz0ii+Um5G9+HNOhKNvIe6dcWi3z1GiXKEgrP8CXhsPVdE/DyHE4SfRqueSEUpuPLVWi2rF29FXXv4mancnDGYEvKDR1o/IOSCWJCAMjl0A9E6xsXaXgzpf01yGQgghDp2MyDmMONx+Hv1mPV+uKqR7spVLRmTx5e4y4t2Trby/uOFG4/XfdvDvCwZyx4erSLDqG92k7Wvpjiocbj+xZiXAUdFMZZxwGOItOubecRwJVgNby2oJaZpPcFcVMlHo8JBg0dfn8gEYkBHDdcd2Iz3WiEatxu/ZikalhmFXwd4Vaz67Xnm6rjXArmVKWfCsUajyF9PNHkLrqow69QoATzWOsJlHTskiUV1Lkr4IKt2w8r2Gdco3wQ/3wMmPgkqj7GhyH7ClSulxcUgsBi0PntWPW2euaDTN0aLX8Ph5A3j0m6Y5rAB+31lFrSeAwdp8IVuH288fRTUR28JhWJVfTddEywH1VbXPcPqgWqckCw5FDigFTQn8tqUcrVrFxSMyuWB4Jsf1SmJTaS3frC5iZX41/zkjg/S0Iah3LYeZk5QRLeYEWPMRoR4TqE09gZuP747TG2BAuh23P8ixSXXo3ztXOZ4n/FM5BxQsVXIy9DsX1n4CX9ykJC4edf3+dgp+fECpXAXK/hx9G4y+GVIHwvW/4CrZRm1lIb7Y7uR7LQz57eXI5xJXOfzxuRKIUsvzESGOBHqthmuP7caibRXKubCuIvrK7mogHL29o9orR85+S4/vseeBlyMP4rvSJ81OIBRm+c5qxvSQ6yUhhGgpEsg5jFTUeusDN1tKa8mKN5NsM1Dq9FLp8pESY8RZqiRBLXN6efDLdTx27gDc/gClzZQk3lfa3lWd9qHTqMiKNZAcroQaF70Menb64gh1GYt6569NX2CKIz+UxNer8zg6N4HMOBMFVW7OG5rB6NwEHvtmPcU1SoLAX67JIbvLMZC3uHHFGq8TvrpVeQrUfRz0OFlZltwXracS1bIZ0GUMrPs0cqd7n0Hqzq9I2/GrkkQ5tT/MuhMGXaw81V/xjrJe+Wb49Dol4fKEx5T1hDhESTYDGhU8feEgdlTUsbW0lpwEC8O6xPH9uiK2lkVOoJ0WY8Sgaz6IcyD2js0k2Qw8dHIGR6WCKhQgz6Xj6UUO1hc5sZu0JOxTbeTLTV569TwT3YbPm25Ya6Ayph+Fjp18vnIXFw7PosjhIRgK89hewakH5pTQb+KL5C64E83OX5XS4FU7CA+8mOqj/s64/y1lZLaVu8bEklG7ApO3CrW1v1IZBuDzG5VAanJfCAZg8w9KIvNBE5Wgjt6qBHW9zqZ9VKmVaZl7gjig3IjNf0YJ0A68CI85jbUaEzN32vnqy0IePzmFo3fOjv6BbvwWhl2pVKoRQhwR0mJMvH7VSHR+B3Q9DjZ8HXnFbscR1lo6X46coBc0Omq84f1XrNpDZwJTrHKu7nocmXEm7EYti7ZVSCBHCCFakDw6PIy4/UEGZMTU36A98vUf/PuCgRzTPYHPV+xi4sjG5bF3VNRxwzvLeH7uFo7vFT3Hy6iu8cSaGsbUJtr0jM5NiLjux1f0JHHtdHj5GHhhFIaXR9Jj5eOozvwPxOc2XllnxnvR+3yySRmN8O7iPG46Ppckm4FxfVL4+0er64M4AO+vc+MfcBFU74jcUVeZEqyJzYLv7oTqPFSrZipP6YderiQ53VdMFpgTUX3zV+W1cx+Bz26A4/4Ovz6t5M/ZZ4oFvtrI2wKlxHnlNuUGMRB95JIQAAOz4qjzBZn9RzGF1W6+WLWL95bs5KQ+0ZNC3nh8d6yG/cfgY806+qVHDiqoVDAoM7b+5xR/AVfsvIdeHx5Hz49PYtzCy3nr2BomDo7nyfMGEm/RM6FfQ8Lv7dV+wifeCwndG29Yo6P8jNd5Yr6SANTjD+ELBkm0GlieV8VlRzVUh6pxBzj/3Xw+6f5PfDcsJnjNXPjLMqqO+yej/7eOY7ra+PfQSnp9dALWL65G89NDqCq2NH4/ZzFsnQuxmcqImP7nK4Gd9y5SgjLH3Bb5wxk6GTbNitw273GoLaG81suUN5fSI9nKC5OG0iUpRrk5iaTfuXDcXcq5qWIreCIEjzxOqNqp3Ny4qyNvRwjR6aTYjWhqiyBrBMTlNF1Ba4Bj/44q0MmqVoXD9SNyqr1hLAc6tQqUvIHVynR9lUpFnzQ7C7c1M2JJCCHEQZMROYeBkhoPm0qc/LKpjON7JTF1fE9mLs3n+3XF3DpzJRcNz2J4Thw9kq1sL3fxzqKGKVZ6jZq/ju9JolXPzcd354V5jW+ULHoND5/dnxhzwxP5eIuBZy4axCNf/cGsdcWEwsp2HjmzFwNKPkM99+GGDQT9qFbPBEc+nDeNUMVmQiUbCCb0pCpxGH+dVcaZg+MZ1jWRORtKCYbCvHLZUP757YYm+zl9SSmXjBhPNmFUaz6K/GGkDVICKK4yJWFp3kKlBOjcR5XqVItegm1zQa1TkhQPuFCZirE3d5VSFWvk9bD0NRh4MSx6saFdo4PMkcr3zhJwVypP90vWKlM1qneC3gLDr1amadiaqXgljmj+QIieKVZuO6knapUKu0nL+qIaftpQxu3jevC/uVvqq7epVXDHyb3ok2Y7oG3HWww8ef5ALnj5tybVse4Y3xOA4hoPqaEyVK+f2jiRd/VOkr68jIcnz8KfnoRZr+Whs/phMWgorqrln8Nq0b89BcY9oIyOK1oF1lT8PU/noR+q+HmrcsGebDNgM+hYmVfNj3+UcuqAVF64dCg//lFCrdfP0bmJ2GKM/FKlZn2hhhuPz+HjBdvxBkL842g7iR+cBaHd0858tWCNEuBSaZRROYMmwnsXKsu2zIb0wXD6M7DweSXAakuFo29VAlDvXxx5W1U7CLsdJIdq+OKaASwrCfLSvC2kxRjpMfhG4nZd23j905+Bis0w8xLlXKNSQc/T4LR/KaXPAcq3wPf/B1t+UG6OuhwDp/5LKX2ukT/DQnR6vlrlb//pT8Oaj2DdZ0oQpOtxynVAbSkkxrV3Lw9OcHfetN0jcrLtB/Hs15YKZQ0jMPum23l74U5c3gCWA3gQIYQQYv/kbNrJFVa7ufL1JWwqaZiGoVbBQ2f1JxwO88MfJUz7dRvL8+KYfsVw7pzQm6uO6craXQ7Meg29U+0k2QwYdRomjsqif6admUvyKa/1MqxLHJNGZRNr0lJR6yXeoq/PlZEWY+JfFwzizlN6U+cLYDXqyFCVo37pmcgd3bkAXCWof/ondWe8zK2/apm7UQka+UNqrh7TlUmjsjHrlaTKkXJ7+INhPl5bw9Q+faJXkBr9Fwjvztux81dIH6pMudi1DD69FgZPUqpcEVZKir99DkR6Sla8Bo6/Cxa/DH3OaFiu1sJF7yh5cfIWKdM7LInQ7zyYdVfDej4X/PYclKxTErlKHh2xj9IaD1M/XMX8LY2rKt10fC7+UIhtZR6mXzGcIocbm1HHgIwYEm2GAxqNs0fvVBvf3TaWdxbtZNG2SlLtRi4ZmcXyndUc/9Q8ju2RyLRBmzBEOpbCYXQ/PYzukneBOFJjjDxydn/C1XlYXxuj/I5/doOSDyGhO7h/Rb11Hn1THuCr3Zu485TeLNxawVuLdvDAmf247f0VqFQqju2ZSIxJx8yleTx4Zj+8wRC9Um3kVbjYUe6iX7qduMKfG4I4u/tDwVJl+uSWfaY47ZgPw69UAkp7l9765SllhM6o65XjPa4LbJ6t5Lzat0TXHsl9UO38Df03t9O160loj3kU7egcft9RycJgb07ofT6mDZ8o6/Y5Cyq3KgHivfu58RuoLYaJH0DADTPGQ91eiU53LoDXxsH18yFxn1FNQojOx5qqPDDatQz6ng3nvKQEmAuXw+c3QFIfwudP71xTq/ZcG6l11PjArD2I3tvTYfsvSk4xYwz90mMIhMIs2VHJCc2MABdCCHHgJJDTiXn8Qf43Z3OjIA5AKAwPfLmW1yaP4Ic/SshJMPOfiwYRtzvPhd2kIzepcfniUqeH6976nVKnl1P6pzG0SyyjuyUwa21xfSWpi4ZncUq/VNJiTQBYjVqsxr1+hUqcylOpaKrzIeDBOv8xBiQ9zNyNcNagdI7qlsDdn62pr5rz4Jl9SYsxRqyMlVdZB+s+h3NfVqY+7fxNaYjNhuPuVKZL9D9PeSqeOVJ58v3bc0rJYXeV8mQelFLDZz0XOYhT/0EGlWHSOcfC2L8p1WxyjwdbupLE780zlCdWx/4NZj8UeRtb5yhVdCSQI/YSDof5anVRkyAOwIvztvLq5cO4deYK/iiq4a0pI0nffcwdLK1GTddEK3ed2psSh5eZS/O48+PV9RWxbEYtuu1zo2+gcDledy16YywqlQqLQYuzbFPjUuG1pco/QAOcOFLDK2YdNx6Xi0YNG0uc5Fe6eW7OZl6YNJQl2yvZVOIk2WbkyfMGkl9Vx71frKXGHSA73sy1Y7uyo6IOg2tX0/4s+C+cN10JHq35GII+JRdOxnDoMaEhn9XeSv+A7/6hfH/ZJ2BNUgK8xpjIiYuPvhX8HjjnJTRf3kqO80q2jpzG0Ow4/vPLNioG38KEoTeQuGsO6u4nwVtnR/7sdi1TqvVt+KZxEGcPvxsWvQATHgdd9LxjQohOQK1VzotBnzIiZ99Rw77aqMnhO6ygMj08rNZT4wtjab5uRWP23aMRK7ZAxjDSY4zEW/T8tqVcAjlCCNFCJJDTiVXUevl0RYSbHZRgzo4KF9/8ZQyJVj2+YJidFS6sBi0JVkOT9fMq6vijSMnr8M6inbx02VDu+Wwtm0sbAjMPffUH7yzK452rR9YHcxrRmZQASrQn3ZZE5WLGXU2MIYxFr+GcIelc/ebvjV4yc2k+k0Z14eGv/2iyifVFNYR6DUTz6XVK9arRtyhBGY0BXKWEk/uiKliu3IwVrVLy45w/DX64T5laAWCOV26erKnR+2tJVEYZjPkrmGLgpPsa2gI+WPRyw7BjnQVqSyLvMyije1IHRG8XR5yyWi+vL9getX3J9kq+v/1YQv/P3nlHR1VvbfiZnt4b6QRC6L333hVFpAqKXbF3vZZrL9feK9IVEBSkSgfpvYTQSUjvfZJM/f7YSSZDJvjde9VL9DxrsTQzZ86cSXJOzn5/7363zY7ZaqPQaMLf49+5i3ZGp9FwIa+cj7ecd3rc112P1Tuu4bA07yYcSi3FvbyIhFBv3PVa3K0u8l/qEOunY80D/bDb7Tz+w1F6NwsG4FhaMbd8u5+uMf5EB3pgs9uw2u088P2R2tdeKjAS5uvGuJZe6OKmQnhrOPmj5ODU5DUsu1XO+5lroCQDNHrwiYJDc0XM2fG26wMLay+F1OZXRIydOB/WPg65p+V5N1/o96icr7s/hrhBMPSfsP4ZWrkVctfWYqb2iOZkVinDOrdC3aI7ZJ8UkbghjPkNZ/EAnN8iYpIi5CgoNG40emh1DRRdgo5TILKbLBQl/Qxn1kPLsaBtIFvvaqV6oatSpcdiA89/x5HjESD3hHlnIaILKpWKNuE+7Dhbf/FCQUFBQeE/QxFyGjF2IMBTT3mVhfGdI2kd7kNZpYWfj2ZwOLUIu92Ot7uOl1YlsS4xC6vNTqsm3rw8ri1tInxxrzP55khqUe3/t4vw5UJuuZOIU8P53DI2ncpxCi2txSNIsiFOr67/nFeoCB+mcsqaDufXVDNj2oez7GB6PR3lVFYpWrWKGT1jmL83pfZ5N520YBWHRhGg0cPB2RDdQ1a+TiyTgqr5cOx9HkRls8KcUfLCDc9Lj7pXqOR5qHXitKksgR53O7dF1DDgKRF/hjwv29fFVCotHjWoNTIFx+6cQ+L0fVH4W2G12ckrq8Jut+PjpsPjsnYom81OcbUr5nKaBXsyvE0YL6xIZNvZXOx26Bzjx8vXtqVFqDc67ZVzCgrKTZgsNtx0avzqiD97L9R3hXSK9iPb91oid73r8vfX0vshLpm82bb9AjEBHkzoEkXslaa1eYVi8PIn3EeE3n9N6ECB0cynW89hNMlq9IGUQhIzSlh2Ty/e2XDG6eWD4/3ooE0l+OJrqLZslwlQHaaIaPvjXdUr3mbs4Z1QHZqPPaYvKksVLBgH47+SoOEWI6RwqotaAwOfhjWPS0tmeS6svE/ybdTaameOHQ7NE9EI4MIWEXInf49nZSFGkz/94oOZ0j0at5prp879imPYcfNtONcHpNjR/DsJogoKClcldhu0GAWRXWH3J7DvK3ELtr8Rpi0F7yZyrWlMWGSaabFVFv/+3+PHQRbJfCMhx5GT0y7Cl0+3nie3tIpg7/oLigoKCgoK/x6N7K+KAoCxykJuWRXH04t5YHA8HaP92H4ml5d+Pom/p44JXaKY3iuGdhG+TPlyD+lFFbWvTcosZeIXu1kxqy/tIn1rHw/3dawI92keyIaTDTtMFu9PZWz7Jk5FIiBF1+g3sZeko8o84njcMxjGfQzrngZ3f9SdbuLC8UsMat2EebtSXL7HOxtO8/lNXRncKoTskkr8PfRE+ruTV1bFq79mcP+1PxBDFqr1T0veRTWq06vhwmbsN69GpfeUwi//PKx+1PkNJi2AVQ/B9J/APw72fSGBzKFtpXjzCIL1T0th1+9RMFWAvtqFpHUHv1jIOiZfJ++QkeeuVt71nhDSssHvpcJfj6ziSn48nM6209lM7xWLWq2irNJCQpg34b7uknPjpqV38yDWV7ct1uXZMa25a/6B2vYngEMpRYz/bBdrHuhHsxCveq8BKCw3cfhSEe9tPMOlAiMtQr14fEQCLZv44OOmI8K/vuvD203HO3vLeXb0VwSuu6fWSg9gbDOFoiaDePGLxFoR5ovtF9h0bwdi296A6sSy+gcx4nUpWKqJ8PcgzNfOd3f05K75B8kqqcTboOWbW7pyLqeM83XE4iAvPa/20RDy/UhHNo6xQIqiS7th+Cuw8wMqhrxGjl9nTF37Epm+DvfV1WHl5zZD036SgxXeSdqsyvOwR/eEQc+i2vF27RQVQFbOTeXww8yGRdiTP0FJJt4txzCpfSweeo1DxAFp8Wozvn4bBUgrppsf9LxH9uOKPg+JmKOgoNCosdutqCoK4LvJtQIIlUUi6CTvghGvYvWJblyjYqs/R4lV7vU8/l3N2S9Grt12G6jUtIuQe86d5/K4rlPE73mkCgoKCn9LFCGnkVFcYWLZoXReXZ1UO81GpYI7+sUxs08sH20+x3sbzjCgRRBNgzydRJwabHZ4Y10Sn07rgm/1WPHW4eLQqTBbsdklMLkhNGoVxRXm6ik7zn/ZLV4RXBg6G19TNoHG82jcfFDZzPDLc5iDWmMe+Cz3rc7ntr5x9IwL4NezeS6dP8UVFrRqFTfN3U+AhwGT1Vb9njCpUzBoPcFY6iTi1GKugF0fyUSqg3Ma/iADnoTld8o3sMstEJQA2CW8NO8MdLxJxJmgFhJaPOhpCfDTe0DfB+HUStnP4YUwca70gtcdj6x1g6lLpY1D4W9BTkkldy84SGmlmWfHtObxH46SV+YQR9pH+PD59K6E+7nzyLAWbDmVg8nqEBG6xfpz6FKhk4hTQ5XFxmfbzvPyuDa4650v3UaThcX7L/HGutO1j+1PLmTiF3v4YHJHxrZrQp/mQUzsGsmQVqFYbXY0asm8+fFEIaWmJjw2YTP+ZWdRmcqoCGrHqvNWInI02OpY5mx2GPvVCQ4+/CpuYe0lf8qYL9OXhr8iuVQqEa4KjCYsVhteblraR/ry06ze5JebMGg1XMov50BKIU2DvEjOl9akO7sFELLzWeeA4xrSD2EZ9Dw7+i7kq91lJGYcp3+LYN7o2UJaFjpMAe9Q+Ga4FA1RPWT0uMEHVc4pccxdHpAMEoyu0TkKr8vR6MFUhnrNo9x+22bs6mKoqHKMIdd7wtAXZQx68g7H63yjJI/Hp0n16OEnYfubzvvueBNE93L9vgoKCo0LlUpaMl1dS3ISsZfnotE1MheKRe4fi23/gSMHJFz+3EZZJPOLwc9DT2ygB9vP5ipCjoKCgsLvgCLkNDIu5Jbz0s/O2TF2O3y5/QLvTepIsJeB3LIqtp3J47a+TWvFmcs5kFyI0WSpFXKsNhtv39iBx384yvYzuYzvHMmhS0Uuj2Fk2zDuWXCIQC89r13fjqgAR993XlkVU7+7QFmVheiAaIbHezM0Vk/I2AWY9b5kGjXMGhLCgeQCNiflMLlbNOsT67t/VCrw89BxW984vtwu2TYatYq5N8bQJfVb3A8Vg7WB4gtQnd8EI15zLeTE9oWMIxDTW4JQ9V5iAc46BonLxZUQPwK63ynCjNkIh+dB7imYvLB6Sk88jH0P1j4huT/L75RC1t0PshLlBiaqh4g4ynjhvw2nsko5klrEuxM78MSyY04iDsCx9BJe+vkkb09sT9MgD36a1YeXVyWy+0IBBq2aSV2jWHbIde4VwJ4L+ZRWWeoJOXllpnptSjW8sDKRbrEBGLRqtGoV9y86jMlqQ69R88X0LkT6u7PxTBEbzxQR7O2DQetHZnEmOo2KDyZH1htdXm6ysjNLzZDe90P7idJWpDWAVwhlVRYKC4ysO56Ov5cbPm5yfYkJ9MDfXYdBo+bXs7kEeRtYfSyT18a3Y8tpCUruHqFHs3tXg59ddW4j36Rdw8nMEt6b1JGNJ7MpVHnhEdQC1fnN0h7VabpMtcIurZP7vpTQYXO5uOZOr3He6Zn1MmHm2BLXb9q0P2SdgCnfo97zCaTukfysvo9IBoZnEPhGwI1zJSerMFmuDz7h8g/EcdNrFrS7Ac5tAptZjtG7ieLGUVD4i6CyWuDitoY3OL8JdfuJf94B/R6YJSOn2CKOnH9byPGNBrVacsf8pB2/bYQv287kYrPZUV9pxVBBQUFB4TdRKsxGRIXZwlc7LjT4/A8HU7mmQxNm70wGZEU+IczbKf+mBn8PPWqV44/o9rN5/Hw0g/cmdSS/rIqWYT50iPLlaKrzVJe2ET5E+LnXjgef/s1evr+rF2E+0rZhB4wmK5VmG2eyy0jJN9K7ZTee23KRTacchWanKD9eua4t2aWVvDyuLW+tO0VplazE+7hr+cfoVqw4ks7otmG0buLN59vO82CfUHqnfob60BxIGC0hxw1h8MYW0Ky+jdndX9oZzBWO8eWj/wU7P5QxoTXknYXEH2HqEllNmjhPsjAqiqRQc/eF9pOh2RDZFjsExUsbWcLoho9L4S/NqmMZeOg1aNUqcktdC42/nMwiv6wlMYGetA734fPpXSirtKBSqfB1110xDNLfQ49OXd+cn1lUgdnqOmS8yGimwmTloy1n+elwRu3jJquN19Yk8c9r2nDPwoOYrXanY352TCsW7b3kcp/lJqucDz7hYDJiLsnGVHCUYquBCl0gnWMDKSw3k1Fcwbc7k8koqmDWoOZ0iPRh6cE0Okb50TXWn01JObx2fVt+ScxGq9FI5oy5vosQwKT34/7B8ZSbrLy2JokYfz2+59ej+vVdEW1D28CBb6VVymYRoaTvw+J6SdsnwsvlnPwJZq6V6XfFac7PdZkp+Q597ofvpjgm3BVckPaJbnfAoH+Ahz94Bsq/0NYujx13X/kXnOD6eQUFhcZJaRYWiwkNdlkUasjd5x745x7X70G1I6fEIqWC57/bWqXVi5iTdUyck0CHSD9WHcvkZGYJbSN8f2MHCgoKCgpXQhFyGhFVZhsZRQ2Py84uqaJHU8fNgr+HHpPFdfbDHf3jCPYykF9eRWG5GTetmiOpRdw1/yAh3gZCfdy4vV9T8stMrE/Mwg6MaBNKkJeBp5cfr91Pcr6Ri7lltUKOj5uOwS1DWHUsE4D7Bzdnwd4UNp3KcXr/w6lFPL38GF9Na09zfSHX3dYEdB7k2n1JyjOTWmAkxMeNW+ceINjbwOwbYgi3Z6FaWz1e+OI2uP4LOLLQ5eezd7kFdf5ZcdCcXAlluRA3AJq0h9T9UJQsgot/U3Hc1BVxaihJhxM/AGr49R2xCTUbCmPflXHneg/Qx4j7RkEB8HLT4u2mJb/c1OA2NjtU1nHJ+brr8XV35E3N7BPLiqMZrl7KHf2acjq7hFBvNwK9DPi46yiuMKP5jZVNq93OiiP193k2p4w5u5JZdX9fvtt3iaOpxcQEejCtRwzrTmSx7Uxu7bY+7lru6R7E4BgNTf0LocQOqLFvfR3d0UXorGY8tW5UdLyV/eHTuHNpCtEBHjw9uhVfbb/AuxvO8Om0zjw/tjWfbD3HXf2bkVFUgadBS0ITb3Ksntg63oR6/1cuP0NOxFBeW3OSW3o35VxOGa8PicJ77YfyZEhrWP8PyD7heEFppjjmxrwj52rcEHHjWUziIEr+VaZeqXVwzUeQkyhBxwYfcenknRZBaPu/HCJOXfZ/Bd1uFyFHQUHh70V5HpxeC9veQFucBg8nSovn7o9db992/J97fL8HNY4cqw6DBnSa/8BBE9AU0g7W5uS0DPPGXadh25lcRchRUFBQ+C9RhJxGhKdBS/dYf5cOG4C24b6cz3XkzQxoEYy3m5Y1xzMZ3iYMg1bN4UtF5JdXcU2HJmQVV7DzfD7RAR408XNn4e09SM4v591fznI8vZgHvz9C8xAv3p/UgeWH0vl+X6rLPJtjacX0ahZUe4wPD2vB5lM5GE1W+rcIZv6eFO7uFUqAG+zNMLPlTAF24IGefgQefB/tvs9FTNHo0LeZRHmr+yHAh1mLDgMwub0/wfveQNWyTgiqqRzSDkj7074vnY7HHtEFVUgrWPeUtEB0nCqrZD7h0voQ3gG2vQFNOkC7iVceD3xiGUz4FpoPgYoCKfS+mwI3/eAU6qqgADC+UyTzd6cQ6e/e4DZeBi1eBi2VZiuFRhMqwN9Tj0ErIbpNg714YkQCb6135N20j/TlyZEtsdvtZBRVYjTZ2H0hn66xAbz4cyIPD23BM6NbUmQ0s/p4Jin5jpHYMYEelFaa602Hq2Hn+Tw02HhuSBPM9ihseg/KKi30iAtg9q6L2O0Q5uPG9zeGErXzH2j2bpUXDvoHZB5BdarOlDpLJe4HPqVbhwpu6TaN2ftyeOC7w3w1oyu3fLuPjzef45oOTbi+YwQGrZp5u5M5ni7uvtkaFT/PuI2E5B2ock85HaNt+Kt4ebhzb0939mcU8+SAMFr42qSdys1XQkXrijh12f2JTI3Z8rq0TtptIsomjIZhL0PKTsnUMniL6LPnM1j1sOxz4nzY+nqDP0sublPCzBUU/m5Ulcl15dd3HY+ZK6Rd++J2xyCEGnrf73AANyaqBexiswZPXQN/QH6LwOZwfgsUXoCA5mg1atqE+7DlVA6zBjX/HQ9WQUFB4e+HIuQ0InQaNVO6RzNvT0q93Aq9Rs2U7lHMmL0PgBevbUOorxt9mgeRkm/kX+tPU15lYWCLYB4ZnoCbRkVKsYkVRzL49ZyjlSM+xIuvb+7C7fMOUlJhpnezQDz1WjpG+VFcYeZSgZGqOi6fUB8DbSN8nI4lJkCyPz7adI4wrZEVoyyEHH8VdVY+U6MGUdR/EmfK3ehj2YP2yHwRcQCsZgzHFtC6Io+LzZ6v3d/EVgb03y2GtuOcvyE735cbpMkL4fwW7BYTtBmHqjgNtrwG13wAa590BBCr1BIw2vchiOkL296UqVXZl9101UWlkRAijU6CT8M7QatroDBFEXIU6hHp787tfeMwW+z0aOrP3ouF9ba5d2AzVCoVL/18kpXVzpvrOoZz14BmRAV44OuuY3qvGEa0DWPX+XyaBXtgtcGsRYcoqg5B1qhVzOgVQ/MQLx4cEs/SA2nsSy7A30PHHf3isNrsvPhzIgathg8nd8Jdr6l3HABDW4XwzjB/vJK+QHNmDRo3X+h9Px5NOtKneRDL7u7Nm+tO8UQvb2JXTZY2wxrCO8KWV13u1/34fCbfeCuz90lI8+ZTOfSLD2bfxQJCvN345WQWTfzca0UcALPVzq3LM1gyZR7hledRnV6NyuADcQNQn1pD4C//YFjCaIYOeg7Nuucg7gE5L/1iJIOhIeKHi4hz4gfHY3Y7nFotgnCLETDgCTi5Qj5Puxul1ar9jb/ptrOrdSgpDwoKfzPKcyXo/XJ+vBtGvib3Ghd3gMELmg2GC9ukBTth1J9/rP8N1W2uJWY1nrr6WYv/L3yjQWOA9MMQIMJNx2g/vv01mSKjqf70UwUFBQWF/zeKkNPIUKtVfDSlE+9tOFubUxMf4sUjw1vgrlNzV/+mXNcpErPVxqnMEirNVuxAeZWFKouN9Sez2XY2l7UP9OOzbeedRByQVotHlhzjmxld0GrUzNmVzKgPZRrL8NZhfDWjK8+tOIFBq+ahoS0oqTCTmFGCChVxwV6E+bqRV1bF08uPMatnEIGHP0Kz99Pa/XtlHsXr2LdETlsGh3bBsJegohA2PF/rttGcXcfQgS/w8ri2VFqshOhyZRU9/QDE9nOeDrPrIxnv2WESRX2fw6f0HJqfH4RpP8CKWTJNpga7TUKL3Xyg590yTeHHu2R61em1rr/h7SfBjnfgzFpZxY8bJLkbauXUUaiPn4eeu/vHcSG/nNv7NSPUJ4O1JzIxW+34uGmZ0TuW0e2a8PSyY1zfOZIBCcG1E6Te3XCaR4cnEOnvgbebDm83HYGeelLyjdz4+W6n6VZWm51vdybTLsKXOTuTOZYuWVYXgUOXiri2QzhzbulGTJAnEX7ulFSa6RDpy9E0R+bV2xPaMzLciNeCYTLmu4aL26DjTXgOf4moAE/uHRBHO49UGe19eq2cr2qtuGEawmbBzVpa++XprFIGtwwhr6yKvLIqHh7agus/dQ429jJomT0+nCbbn0bdbrxMhMpOlBYmqwhY6tNrZBpcy2vESdPmeri0R3KrGqLZIHHRueLCVuh5r2TeTF4o14yqcpjyPWx7S3KymvaXVXYXWGP6yB9RS5WEHVeVgs5DQpAN3g0fk4KCQuOlLKv+dD2tAWJ6wYr7wDtMFn1Ks+DQPLk+3P3r/+ZY/xsslaDWUGxS4fHvBh3XoNFKe1X6IRHJgU5R/nxtv8i2M7mM66hMr1JQUFD4T1Gq0UaEyWLl618vsjkph+k9Y3hwaDx2O6QVGnl9zSn6xgdy94DmPLT4MIdSimpfN6BFMB9N6cT93x2mymKj0myjpNLCuhNZLt/nfG4ZHgYdU77cQ26ZI7hv5dEMdp3P48PJnai02HhkyZFahwBAVIA7C27twdrETI6lFRPv7uYk4tRSWQxbX4PBz8G+L6T4GfYSrH+mdhNjfioZRdEMbhVCJWV4AOz9Em78FqpKIPOoY3++kdh7zkJTmoZarYHxX0pvd6nrz8ehuXKTFdtPVuWNBRJafH6T83ZBLRxtWCCr+Oc3Q1EK3DgfKoolwBTkRs1cIcWn5t9NBFT4K6HWqPho8zl2nstjXMcIPprSGZvdjsVqY/nhdMJ93ZjYLZrnVpygoDpLx12n4YEhzTmdVUq4r3vtNI/kvHJ+OZntJOLU5ZMt55nYNbJWyKlh5dEM7h7QjJhATwACPA18NLUzt87Zz7mcMu4eEEeA3oJ++2vOIk4NRxZA9zvQqXzonL0a/cX10sI0+m3IPwtb3wA3P8f2HadCq2tl4ptGB6XZuHkFAEWAOJWaBnvy3JhW+HnqyS83SWByHZ4dFEL8jofQZOyH7rfB3i9cf4OTVsgUqIxDkklR+qRkVjUUlKzSiojbEFWlItLu/QI6TJWR4aselNZNjwAY/xVkHpNWq7r0fgBLaT64ZaM9OFtW6M1GWY1vORZGvi7T8BQUFP5aaF20ztrt0O9RSN0n9x51F4d63iuulMaGpRI0Boqr7Hj8N9VCcAs4tUYckHpPAjz1xAV5svFktiLkKCgoKPwXKEJOI8JosnIstYj0ogreWHeq3vNtw+O4f9Ehp1V3gG1ncjFo1dzUM4Zvfr0IgNlqw2aXFo1+8UHEBnqSW1rFxqRsusT4s+pYhpOIU0NemYl9Fws4l1vqJOIApBZU8OjSowxpFUK32AACs3bUe30tF7bIVJjwTnB2gxRMXiFQJqHIBu8gzuWWUVJh4eG+QbKqrvOAo9/L1CmfcBkd7hEExnxUCyfg02UmtBoHZ3+58sjvqlJxFOz7QgrQdU/JVKr2E+HYYhFl2k0Ar1BZob+c/PNQfEmONboHFJyXfvn8s9CkM/S4Q9o9jPmyTWUx+ETIRKsa4UfhL4vRZCUps4Qqi40lB1JZcsDRjqRRq7h7QDNu+novFpsjc6DCbOXNdaf5dFpnSqvM+LrrKTKa2JCUTWqBI+9Gq1YxICGYKH8Pskoq2Xkuj5DqoPHL2Xwqm9bhjrbH6AAPvrujB5nFlbjpNJTlpKA/s6rBz2FP/BG/gosinNRwahV0vVVaGg0+ENwSutwi58Dimxyr1P6xBE6YS4tQL87mlDGqXRPuWXCQFbP68NSyYwxuGUr7SF+OVV+rVCroE2ZDs3WPCETG/Ia/wTar/Nv0kkyhm7QQbGbskxaiWnyTo1UTIKqHuGOuhMFL9lezYuzuJyIOiMj1y7Mw4Rtpj0jbJ9eFNtdD5lHcji/A7hvlEHtBRKOklRK2PPk78Aq+8vsrKCg0LrxCpLW6NNPxmNkIPz8kwm/KTri0GzwCoe0NMv2uqqTB3V21mCtBK0LOvz16vC5BCWBbCZlHIKYPAJ2i/fklMQuTxYZeW38So4KCgoLCb6NcPRsR7joNTYM8G3w+JtCznohTw4akbHo1c0y0ctdr6Bnnz+ybuxId4EFSZgkGnZpPpnXmhs6RbEzKbvB9fjmZTWyQl8vnDqQU0i8+mOfHtkbzm79ddtjwghSGSauklzwoHoa9TIXGCze1jX3JBVSaTNh6PySFo9YggkjqfgkhXXYbrH0cutwqoaPmchFimg+Dof+E0Lb139bNV5wDuaeg5Rjo/5iIO6HtZIpVWFvwi4Xvp7p2K3gGQUmWjC7OPQVf9IOj30nxt/9LWHqztIR8Mxy+HADzroWPu8Cax6G04e+rwl8Ds9XWYNhxv/gg1idmOYk4dZmzK7l20lyV2cbR1CLiQ6VFp0/zQL65uStNfNxIyizB26Dl02mdaWhgldbFmPJgbzfaR/rVGTN+hQBLSxXo3GSVecgL0Ga8uG0OzJab8apSuPZjccHs+8q51aAwGc38cbw/MoiXx7Xlp8PpaNQq8sqqOJJazPf7LnHPwGaoqo/dXadBa6yebGcqvbL4otaCtUraHosuyflvs6I6+j1Mmg9j35cg5onzYcBTIrzGDXK9r04zRCAe844IUlYzFKU6b5OTBAsniAswvJPs+9xmaZloMQLVkQXiwrmctP1Q6nr6mIKCQiPGuwlMXiQO3BpMRllcWjgBck/LPYh3GKx+VCbflTXCv/2WCtDoRcjR/xdCTs1CXY1ADnSL9ae0ysKeC1cQ7RUUFBQUrkijF3I++eQTYmNjcXNzo0ePHuzbt6/BbRMTE7nhhhuIjY1FpVLx/vvv/3kH+jtg0Gm4vV+cy+fcdGqqzA2H0dntOI0id9dpuGdAc+6cf5B5u1PYe7GA5YfSuWPeAZr4GvAyNOxo8XLT0jxY7LGuMFttaNQqKmMaKJ4Amg6QP+qWSnGsaPTQdoJMoUr8kZCfJvOO/zLWzWxKk3NLUe//EpJ+hsMLYO5Yycvp86CILrdvhNDWkp1hs8p0hEUTRVxpPwmu/QjUGtB7yWp/11tlxWzoS9J2EdxSBBZLpUyyOrIIKgulwAN5TXgnaDESbpwrApHNBPHDpIAzOIc90/8JuZkrvqwgPL5Y2jeszk4mhb8OaYVG7llwkCndo10+H+7rxtns+pPfariYV06NxuOu12CzQ5twHzpH+TKlWzS3zzvAgr2X2HuxgKUH07h59j6qLDa6Nw2ot6/BrRrOjdFp1Kw5V4EpfkyD26hajqmewrINDs+X1qWpSyC8s5w/Wjdx4uz93PUOKotoVpXEoZRCVh7NINzPncSMEnQaFe0i/Qj21LHxkQF8e0s3HhuegNq72rlis0LBBTnnXNF2PJxeA6Ft5GuTUc7BzjNA5wmR3eW41z0NPz8A/tHi4usyU647nkFSVNyxFTpNg+IMCSYtz4eo7hLiHN4JAupca+12EXIC48Vl16S9hKkHNpfJV1O+h36P1W+rLLjQ4PdXQUGhkaJSyb3CPbvh2o+xdb8Lu94TghPkWtD1Nrl+tBoHI14XIcPzCjleVytmaa0qMdnx/G/9+0EtRNyuXjyIDvAgxNvA+sQGWuAVFBQUFH6TRi3kLF68mEceeYQXXniBQ4cO0aFDB0aMGEFOTo7L7Y1GI3FxcbzxxhuEhYX9yUf7+xAb5Ml7kzripnP86AxaNQ8MjsfTreG/tGoVtfbV1k280WlUvLAy0WkCFUi98tyKRG7uFdvgvsZ3imD3hQJeuKY1Dw6Jd3rOXafBx03H9G/2seikmfKOt9XfgcEbet0nORwANgu2XrOwH5wrrpWMQ2DMR9+0D5q0/WhyT4qYct2n0P9xec3Jn6SoGv2OtDwk/igOgtnDqnNv8mVVbMNzUJAMM9fD2Hdl5b3DFOhxtxSiK++TwOPji6sDDK3iMvCNgR53SbE29l1p7Wo5Vlbgji6GtU/A4mnw84Nww9dS5IIUiFVlDbeG7Pui4ewehUaN2Wpl3u4UjqeXcDa7jAeGNEencaxieuo1XNcpgnaXTXmrS3yIF+46mTDl467j0eEteHfDGV6+vh2vrknCbHV20Njs8PLPJ7mrv7PAe1f/OEKrW66sNjsZRRWczirhYl45xRVmgrz0rD9TSlbXx6U96TLs7SbJCvLP1VkxBRfgyEJpnxr0NHaDL2b/ZiJkdJzqLHrU4BuF1VKFj7sOrVpFSYWZbuF6tt0cxqfBP9J1/yP4nVlGoDWHjUnZePiFOhx02/4lzpeY3o79qdTiCmozHtIOQWALuHW9nH8laTJJxlQmwtLZjTDjJwkyzzsPbt7gFyniz43zZFrdjndgwQ2w8TnJsmkzTsYGWy0QNxC63QZTF0uWls5DsrcOfitC8prH5HsxZ4yc+xtfgJxEuO5zZ3eOMtlOQeGviVojU+06Tyez1z/lvuaGb+X8Lzgnwm9pJug9YPqKxinkWCpAq6fkv22tAghuJUH51VNEVSoVXWMDWJeYhbUBh6qCgoKCwpVp1Bk57777LnfccQczZ84E4PPPP2f16tXMnj2bp556qt723bp1o1u3bgAun28MeBm0jG4bRtcYfxIzijGarHi76Vh6IBWVCrpE+3PwUv2Rx2PaNcFqszPv1u4khHlTbDSTnG908Q5wPrec2EAPRrcLY81xZ9FhaKsQLDY7i/ensnh/KjP7xDKjVwzzdqcAMlp5x7lcskoqeWt7JYEjZzBu4jAM+z+VNqXontLOtOEFUOug663YwzthrSxHd+pneROVCq77TEYGZ9cZK3xkoYwJvu5zMHiK42XrG1K8DXsZtr9Z3+3S6hoIiIX54yRoD+QGrNcDso/sE/JY1nFppRrxGoS0kq/bT4LldziPNtYaxOEDMj0r84i0WHSYCge+kZu14ksN/wBNZSI4KfzlKCg3s/KItNJ8tu08Y9s34fObulBUYUanVhHopaddhC9BXga++TXZZYDxrX2aUlJpxsddnB0Jod7c0S+O7JJKMosrXb5vaZUFL4OWiV0jKDJauK1fU1qEeOPrriO7pIItp3J5a/3p2mDlPs0DeffGDrx+Q3vuWp3IJ+NXE3zuB7yTfwE3X0w9ZqENboHq0+7138xUDidXQu/7se78EN3FDSKy9rxXCpi1j8vXI98AsxH31F087naBW28ey8E8De2Lt6Bddb/syzeSwMoSAtU/89qAFyittOA9/GW5NmQdk7bJIS/Ivspy5TVZx6EkE8Z9JOPC3f1FrK0JOVZrxanX6lqYe41MnAptAz/dJc8bvCFhNHw91JGlYyqTyVjxw2HXJ87XHK1BhN2EkXBwbv0x5xWFsOJeGPUvOV7fKBkxfGq15Hj5uXZmKSgo/HWwWsygs0DhRfjxTse9BojYfe3HkufX2DBXYla7YbTw37VWgYheOjdIPSCLcEDPpgGsOZ7J/uQCesYF/sYOFBQUFBQup9EKOSaTiYMHD/L000/XPqZWqxk6dCi7d+/+Hx7ZH49BpyHCz533N55hzfEsKqpbqmKDPHlmTEveXn+G3dV9xyoVjGgTxh394/j1bB59mgehUakanIITHeDBuI7hqFQqOkT6MaZdODvP52KzQd/4INILK/jnysTa7b/dmcy3t3Rj3fEspveOYWz7cMZ9ImM27XZ4Ym06P8cH8fmQF/G8sFaEk4UTYcjz4BmEPXE5qsML0NRdxY4bDCm7nAuq0DayQl9wAdIPys2Rf7TkcuSdkcyM9EPOH0atlXaLRZOcp9bYrLDzPXHb+DeVm68aNr8C/rHSPnF+c/3CzVIFK++HG+c4xqCfXiMtV8eXQNwAaD4cUMHxpfVbKzwC5GZG4S+HCgkzrmHVsUxWHcvEXafBZrczuGUI3ZsGEunvztxbu/Pg94fJqc6q8TZoeWpUS7aezuH5FSf44Z7ehPu54+WmI9BDh9sVWh0B8sqqmDUonlAfA2462bagvIrtZ/J4arnz7/DOc/kcTi1i8f5LPDmyFSuSi/B0n0KPwTfh7enOshPFPJK7wvWkJ4MPtLke1TfDcKs7fjxtv4img58TV836pyHvLCrAE/Dc8zaRd2xF/fWDkhsx/FXJ2Mk8Al6hxOhLsHj6wne3Q9+HYfA/QKWRsNDZI+pPo2o3AbreDnNGyYWmBptFnDYhreX17SbAwhsdz1/7ibQ3mi8TsdtOgMPznK854Djf79oBvzzn+ptvLJD3dfOVtsxrPoCMw3DTjyLmKCgo/KVpoilBZTLCT/c4izggDpRf34VRb/5vDu6/wVJBqUoy2v6rqVUgC2iBLSB1D3ScAkDzEC+CvQysOpahCDkKCgoK/wGNVsjJy8vDarUSGhrq9HhoaCinTtWf6PSfUlVVRVWVw0FRUnJ1TB6otFi5s38cgxJCKDdZCPY24KHXUlxh5v4hzblrgLQ6+LrrWH08kwmf7RbxZv1p+jYP4sVxbQj1MZBdIp9NpYJnx7TCoNWQXmhk9s6LLDmQRqdoX94Y35431p7mmeXHKam01DuWC3nlfDClIz5uOqw2W7381B1n83jTJ4T7Oo4gKPc06hu/xW63oyo4j6qqFLITUdfkXQC0HA3b3nJ87d9UVuaX3yF5OjUYvGVCxC//kK9Vaufis9kgOLO+4dHDB76FDpMlNLkGs1GKMp9waZdwhaUKClNE8ClMFmFI5w43fAMH58CS6dJS0ft+abPaUKcA7PeY0m7xH3K1nos1BHrqeXhocyx28DbouFRQznf7UrlUPXVqWs8YdBo1+WVV+LlrmX1LNyrNVsxWG2VVFhLTi0kI82HB3kvsOJtLWoGRcrOVse3Ccddr8PfQUXjZpDiQfCy9Vk1iejExgSIcWG120gsr+HDzWZkI1SyINuE+lFdZWJeYBajYcjqPLafz6BDpS4iPGz8dreBkZgkdIn15uIOu3vsAcr7s+8L5PKwh6Wdx5hz/AfLOXvbNaY764jZpTRz3Cax6WIKKq1HteBvtxAXYJ3+HqjRTHDjmCvj1Pdcjxa1mycCyN2DJ3/819L5PBOEaEkZJ6+P5jfW3bzFCHDWusFRKu6dvpNMxO2HMk+tRcZo4+m7f9JcWca72c1FB4c9Eix0KLjY8mercBjC/+Ie89x96LporKUYGW/zXrVUgeYTHl0BFAbgHoFKp6BEXwJrjWbxwTRt0vz0hQ0FBQUGhDspV8zd4/fXX8fX1rf0XFRX1vz4kUgvK2ZCYTXKekaZBnpzOKuXOeQeZ/OUebp1zgGeqV+ANWjXXf7qLr3dcdHLg/HoujwPJBTwzulXtxJu7+sdxMqOEZ386gUqlEnFmckfGtAsnq7iSLadzXIo4ghRTM+fs51xOGY+PaEGEn2TG9IvzY83NMcyKTUVXdBHVkOewp+5D9cs/JLg4th/0vFtElxo0Bufirff9sPbJ+sVjValk6vR+EJJ/lfDhuniGNFx4gbRAebnoW1drJLS0JvfGFcY8R8ixu7+MF/9+irhzSjOl8Fv1sITBdr9T8ngGPCkjztWahver0CBX47lYlwKjmfxyM2+vP82sRYdYeTSDh4e1YHK3KIa1CiEh1Juc0ko+33aeaz7eydiPfmXC57t5avlxtGo1Px/Lwmq3ExfkyZbTuaQUVDD712T2XMhn1dEMXry2Te2Up7o8MDiexftTuVjdKmk0WcgsqsBis2PQaphzSzdaNvFm78UC0goreHp0K7zr5GkdTStmw8ls0gqNDEoIIcTHDXvzoa4/ZExvOLuh4W/C6bXODrcatAaZKNdxmjhiLj8vbVZUP9yCqiJfpr5tfR0iu0qosis8g68cJFycKq0Mda8ZHafJ+VgTYn45Gr3LvCBAHHo+EQ2/X42LzytErl2NsY3i3+BqPxcVFP4oTBYraQVGzmSXklpgpMpSvZBTUaelXe8pLaY12O1g+2OGHPyh56K5giLEkeP137ZWAQS3AFRO06v6Ng+ioNzEjrO5//3+FRQUFP5mNFohJygoCI1GQ3a280jH7Ozs3zXI+Omnn6a4uLj2X2pq6m+/6A8ks7iCC7nlzN+Twre7LrIhKZvZO5Odxhkn5xt5ePERNC7GD9fw7oYzeBm0fDWjKyPahNIrLpBlh9IBKDSaeGBwPF/tuMArq5M4kFJI15gGChygV1wgey7k8+K4Nmw6lcO+i4W8Pr4dWx7uzVeDzLRePozQtbfj7x+Aas5oVLs+lEIu/5yM5dz6pogiHadVf8gjkldRg3eY6+IQpGDzDIKj30P3u8CrjkOr8KLr8eM1hLaVVbTIbjD2PZg4DyYtkKBjkGyOqUukPcPVa2uOqdcsqCgWZ87lHJwDnW+BWfug3yNSgCr8R1xt52JdCstNvPhzIq+vPUVemWTRJGWW8vDiI/RuHsibN7RHrYIVhzP4asdFp/M1Jd/Iw0uO8NiIFiw7mMbYDk3w99DhZdByz8BmtI3w5ZOt5/F11/HV9K4MTAgm0t+d3s0C+WhKJ3LLqtiYlEO7CF8u5pbx9PLjnMouJaO4gufHtuaB74/w9Y6LHEktYuuZXB5dchS1SoVPtZijUat4Y2Q4aye483nAQj7xnY9Gq4dut9f/oCqN61HbNWh0rjOg8s9Bk44SInzOhSMG5HXF6eJkKUkXd0tD71V4seGpViCtmNknZApVDXab5Opcfj7H9IGAZnDNhzKRbtpSmWxXl6AW9YXiGqJ7Qf4FsJqgxz3SntkYRw3/G1zN56KCwh9Fbmkl7204w7D3tjP8ve0MfXcb3+y4KG3cIa3lWjJpgdxPDHtJriWdb5Z7FL3XH3JMf+i5aKmkqNqR49WASfPfQu8pWTmpe2sfig7wIDrAneXV958KCgoKCv9/Gm1rlV6vp0uXLmzatInrrrsOAJvNxqZNm7jvvvt+t/cxGAwYDIbfbX//DTabndzSKmYtOkxZlYV/TWjPG2tdt5EVGs2k5JcT7utGhouQ1CKjGXedhrvmH+SO/k05lVVa+9y5nDLSCo2cSBeL7vf7UnlzQntmLTxUm8dTwy29Y6iy2MgpreKDTZJR8/iIBHaczePGZlbcfpgoBVrLMRIAWuZioljWMchOhIguEDcAe2UpqrA2cGadBJHaGnICVWM1i6V59SMwca5MtEneJvkabW+AfV/KfsI7Q/fbQe8NarWssBdckGyL7f+SYNSIbrDsdmmnALlB6/sQ9H1E+txB9lOeJyv4Q56XArH8CoVb9gnoMOnKn0HhN7mazsXLyS2rYtWxTJfPvb7mFD/c3YukrFK+3O7aRVJkNJNXasKgVeOp19K/SzBJGaUk55ezMUl+t97feJbrOkcQ5e9Br7hAckqr+OfKRPLLTUT6uxMV4M7JjBJGtAnDoFWTUVjBsoNpFFfUXwl+f+MZXrmuLQ8uPsJ7YyMYdul93Lcud2zQfJCcPzd8AyeWyWpzdC9pL0oYA0krXH8jEsY4tzPV4BUKHoHY1VpUDbU6gjhoahwzyTtE0D2zrv5257fA0BclpPjy1iuVSlx630+BsR9IOLlvlOy3y0zwDoVL+yDzMLQeB+0nw7cjxeEHIh51mQkjXoX1/5DrUpGEuTP4WdhTHdyu0UOb62Ufqx5yTNS7sAVbWQ5qv7+uS+VqPhcVFP4IyqssfLDxLAv2OtyEVRYbq45lck/vJuDmD11ukSmYNTk5KrUI4pMWgM7zDzmuP/RctFRQbJfj/l0cOQBBCXBxmwjfGj0qlYp+8cEsPZBGkdGEn4f+93kfBQUFhb8BjVbIAXjkkUe4+eab6dq1K927d+f999+nvLy8dorVjBkziIiI4PXXJQPFZDJx8uTJ2v9PT0/nyJEjeHl50bx58//Z5/j/Ulpl4afD6VSYrYxoE0pCmDePDm/BznP5rE/MclrlB0jJLyesjpDTKcqPmX1i0Ws16DQqwv3cmHdbd8oqLVzIK6t9Xadof344mFb7dW5ZFR9sPMMX07uw+ngmB1MKCfTUc2vfprRp4s3ZHMkCAWgW7IWPm45lB9O40y/FsTofNwh2vt/whzu7HrzCADuqrrfBuqdh0nyZFKP3ktYMVyv9Gr08ByLo2GwQ3VtEmaJkEXdu+BqSd0JwAmx6yTH+e9CzsvJ/8FuxRre+Dr6b7Lx/mwW2vy2jh/1ioOkA6HmPOAJGvAqHF0ob1bhPGv5smt9jKUvhauZEev3MmN7NAhnRJgw3nZoqiw2L1UZuWcMTy5Lzy+kXH0z7SB+sVjvB3gaiA9xx12kY1CKYSqsNb4OWtuG+fLrlLJtOiRW9Z1wAr17Xjnm7U1i09xJVFhtf39yVIG8DW0+7tqvvvVjAzb1iWfdgP0Kzt+N+arnzBql7Ie+0iKAtx8j0pbR9ko8zaQFc2iXjvuvScaoULr1mybY1DjXvMFmh/mEmqv5PyL4aankMaSUuOxCX3cS5kH5AhNO69HtM9jHtBxmRXj3SFp9wEVu8w2DGSnk8JwkqiiAoXoKQvcNk6lXmMWjSHr4c6CwW220ygW7km9D/CYjoJOKuqVyuKyNeE4dOzecry4ZRb8nxVotOdjuUVDimjykoKDRu8sqq+G5/fbdLpL+7DFuwGOGnu52duXabXEOjezqcvo0JcwWFdk/0atBrfichJzhB7veyT8iiGNAvPpjv96fy0+F0bunT9Pd5HwUFBYW/AY1ayJk0aRK5ubk8//zzZGVl0bFjR9atW1cbgHzp0iXUddqLMjIy6NTJYcd/++23efvttxkwYABbt279sw//38ZksZKcb+Sbm7uy8WQ2t87Zj80OQ1uFMvuWbjzz43HSCh2r063Cfdh9XqZXDWkVwuh2TXh+ZSJFRjMPD40nKbOEL3dcoKzSwjc3d6t9nQq4TBPiWHoxa45nMqlrFFO6R+Nl0HAyo4RDl4rYeS6/drsbu0Qyb3cyvu469GVpUjz1ewwiOsPuK4kdBhFlWo6RwqgoGb6fBgOfEmdArwfAVCKtGRaTiDcXt8t/Mw6JKJMwEvLPS87FqoekFaL3/VCUBq3Gyqp7TTiqSgXhHWDrq/J1i1EyVrkhDs2HCd/Cnk/gy/7OY857PyhTalyh1krA329Rni+5O5YqcPeTQGRFAGo0+NYp2L0MWt6Z2IHDlwr5fNt5yk0WBrUI4fb+cfRtFsiv5/Nd7iM2yJO+zQPJLKrkyeXHSK7OvIkL8uSJkS2ZtzuF3efz8TJoefHaNtzZvxlWu50mPm68v/EMK446HEEVJit6rfry3HEnLuaVMzrBG375qv6TRxbBhG9g6S2SZQWS8xTUAlL3w7Ql2M5uQn12vfy+tp0g2VAnVwB2EXt2fyLunP6Pwy/PQkmGCCT9H5dJUJfTfIiILjWCbVUJHJoHM9dLQOal3eAeAG3Hi6vOPxZMpTB5EVSWQHmOiEQ2u4guiyY6tzjpPeH6L2DrGyIU97pPjrchx9+Bb0SgnTPacb5f3C7i0IyV8O0oEbPcfCVI2T9GhJ6cJCxqAxdyy+gY3XBLqoKCQuOhyGjGevmNEeCh18h14PgS1+3VIItBU5b8wUf4B2CppEjt/vu5cUCEdHd/ycmpFnJ83XV0jvZj0b5L3Nw7FpWrMDgFBQUFhXo0aiEH4L777muwlepycSY2NhZ7Q1NOGgGeBi139o/jsaVHnQSbJQdS2XI6h7duaM+tc/djt0OApx4fNx0PDWvB0gNpjO8cwW1zD2C12ekc7YeHQcurq5Nq97EuMYt7Bzbj063n2Xk+j1Ftm3Aktaj2+ZfHtSUps4SJX+yudf7Eh3jx1Ywu/JLoKJZCfAxcKjDiadBiihkITbvIWPCcJGlDqGlPupz2E6VIW/MoDHsZ2k2UtonQNjBnDNy8CnZ9IOKO3SZCTMtrZCR5VTH4REn7Q0BTmYjQ9gY4MBuKM6DPA7DlNecJN25+UljWPFbjOGiIoktSsBanOYo6jU5yfcLayI3JwTn1A5mHvuh66k5d8s/JRK6a8el6Lxj0DHSYIuPKFa56EsK8cdOpqTTbeP6a1ny0+WxtayLAiqMZbEzKZuEdPTn01R6MJucbfn8PHd1j/SmpsHDzt/swWx2/qxfyynngu8N8fXNXDiYXUlZl4dGlR/nm5q646TTMnLOfewY2dxJyvt2ZzJOjEhjVJowEfzutArWkldn49lARqQUVhHgbmNDeH7OxBF1Ac7iw1fkDVRbBxhdh8neQtBIC4+XcyjwGeg8RHtuOl997z2BY+5i0TXoEwvWfw6pHoN2NkjXjGyXnP0DWcUjdJ9Pmdn4gq7Lu/tB5BjTpIG0Jdel2OyydIedEWDswV4JHMGCTaXAZh6Rtq+ttEsR8Zr2sjp9cWT+nxlQu7p2Rr8PyO+G6T+HY9w3/UItTJbsroBkENhOxyFwO130uz498Q9pCwztD4nK5RkR0hRGvYSwpZsERCy1CvfH4jdHxCgoKVz8eetdDCuJ8qid1Xj6pry7FaX9Y2PEfirmSYp0b3r9nt5NKJQt8qftkEEQ1Q1uF8vraU+xPLqR7U+W+R0FBQeH/g3KH2Yhw12k4kV7sJOLUkFtaxa7z+QyIDyYh1JtxnSJILzRittkZ2z6MradzaleTJnaL4r0NZ5xev3h/Knf0i+PTaZ1ZezyTwS1DWHEknTPZZfRtHkR2SSUL6/SGD2wRzC19Yll1LJOBLYNZdVyKyLTCCuJDvTiRXoKbTwzMmyoF0+JpMGmhBALmnHQ++JZjJXujPF9W4435EhqoNYgzoOtM2P6WTISqwW6XAtNSKQVVyg745TlH8RbTW1qzLKbqDJ4Tzu9pqZIAwho8gyGsg+t8D5AiUq2DFiMlxLCiUASlU6ul+AxpI3kiKbvEneMbISHHhclweB6UZkjB53vZ5JviNJgzVtwMNZjKYP0zImx1nOL6eBSuKkJ8DHxxUxf+8eMJ7HacRJwayk1Wvt5xgVeua8uTy47VijVNgzz5ckYXbDY7JZVmPp7amW2nc1l2KI0qi+TJmKw2fj6WwdDWIaw5Lq2BSw+k0TTYk+R8I/4ezu6tQ5cKuZSZwwf9rKi3vIrm5AnwjWbskIcp9kmgifkSnmvvQFVZiL3ZEFTTfoB1T4moaPAR90rOSTizVgSbXR/D2scdb6BSoR7xhoin0T1EGC3LkXN36xtw7YdwbAkcXyptR3U5PF+mzHW5Gfo/jj24BaridFgyw9np1nGauICyE+Xr1L1y7plKRNCtcdKUZMDK+6DTDHH1aHQi8LjCmC/OP41eRpxHdIXEH11vG9oGsEtb5Z4vxFnXdgJVnk1QVRahL8+V79XSmx2vSdkFh+bhPeNnMouNVFpseBiQ1q6SDMkbqiwSETqkpaxOKygoXPUEeulpH+nLsTTnxZowLy0qdz8I7winVrl+cWgbyeVrbJjLKVK7/T6jx+sS3FKEnOL02nuithG+hPu6MXdXsiLkKCgoKPw/UYScRkRJpYU1x10HqgLsOp/LR1M688X2C1z3yU5MVhvuOg039YymV7NA5u5OwW4Hb4OO7BJpX1CroH98MNGBHqQXGVl+KI03bmhHidHM+5M6sedCHuF+7jxVPdIcoHO0P9d2DOf2uQew2Oy8M7EDrZp4k5RZypIDqTw8rAXvbTiDKvu4FC02iwgvP90LY96WYurML6DVi4gTFC9/0A/Ph8mLsFWWoDr1M6qWY6XwaT8Rdn/s+kOf/UUyMZbf6fx4yi7Y97UIRDo38ItyzvQwG8EvVgrQyiJZfW91reTl1AQd16BSi5hUkiktWGn74fRqRwhrVHcpOk3lcsMWnADBraT1wlwdenhwjrh+bl4lLRg1ZJ1wFnHqsuUVaDZQ2qwUrmr0Gg094wL5cVZvXqnjdLuczadymNg1ik+ndcFkseFh0BAT4MGqY5l8tf0CpVUWtGoVI9qE8cm0zjzw3eFa987JjBIGtHBMPbuQW8aELpH8eG9vzFY7S+7qSUq+kf3JBcQHezLS4wS6b+uIDOV5hGRMJ7jfY6iKU+H8JgBU6YfE+TVpkYgx1ko5RwOaQ0AsJP1cv0Cx22Hdk3DTcriwDQY+LefUiWUivpTlQKtxENYWvIKlxbBuC1PhRdj4T1BrKL31V3yOLYEbvgJzlYQR6z1F0KwJDa0hbiD8eLfrdqjD86DbbY5g4oaoKhORuCxb2i+3/0uuAZfT71HY+aFsFzcANjwPv76LfczHWCJ7oYvqjurrIfVfV1mE+pd/cFfP9/AyaETEOfCN5HPVsP9rEZEmzZdcHwUFhauaAE8DH03pxM2z99W2vQJE+LlhP7UKVVQPEXar6ov49JoFqkZ2u223g8lIodYNz9+ztQrE5ajRQfr+WiFHrVIxvE0Y83enkF5UQYSf++/7ngoKCgp/QRrZX5a/N1o1uOlc23sBbuwSxZvrTrE+MZuoAHdCvN1IL6zgqx0XqbTYGN8pgmWH0rHabHjqNbQO9+H+wfFsOpXD6awSIvzceWtCeyL93DieUcLLC5KIDfTgH2NaUWR0rJTf2jeWp5cdr22xenFlIu9M7MieC/msPJrBxbwyvprRFbdTn0lR5xUmo73tdsmBubC9OsvCDJtfliBUm03EkG+Go7ZWwZTFcHGHrFibK+W1Gr1kUfhGyur26TXVI4vTXH9DdO6QewryzkhRll5nld7NT4rJse9JQGHGYVkluuFrKTBrwlN9I2Hwc/J1q3FS1EX3lEk25zaI26CiELa+LiHK/k2lLer8ZoeIU0PRJckKue4zMFSPIm3IOQDyucwNh+MqXF0YdBqCdRp83BrONnLXaQj00rPvYgGxgZ6oVbD8UBofbzlfu43FZmf18UyySip5dFgCPx/LwGa3E+7rTkG5qXa7J0a2ZOf5PObvTqk9F+OCPHnzhvb4WXLxXvF4vfcHUJ1aJS6ThNEiUl7cBuV52O02VKselN/nGsZ/7cjIcUXSz9IGOXc0TFsmgeBWk7Q91Uyo6zxDgpAPzav38op2M9ibo2aI1h31jnclQDxll4ib5zbJ+Tj4OXGv2GxyHbjc0VeXtAPgG95wQaVSyTXFN7L6azVMXQyrH3W49jyD5Rx2D5LWK7tNhFy9J5jKcVt9P0z/UdrFGmjVVSdvp8doFXqtBvJSnUWcGtIPSPZWv8dA0/B1XUFB4eogJtCTxXf1Iq2wgpT8cqICPOjgVYJq1XJofb2I0RtfdFyjvEJg4DNQWYpdY6BRJb9YKgE7hRY9gb/35HStHgLi4NJeGTJRzYAWwSw9mMqcnRf5x5jWv/ObKigoKPz1UIScRoSnQcfMPk3Z1UBYaucYfxYfSOWL6V3ILK4ktcDIDZ0j8XHT8u6GMzw9uiXLDqWz+ngWd/aPo2UTH+6Yd6C2fQNgzYlMFt/Ri8d/OIbdDkfTikktrMDPQ0eR0YxBq8ZulwlaNZRUWrhz/gH6NQ/ioSHx9GkexPzdyTwc1xGvSfMl6+bkT9I24d8UBjwpIovVDJ2mS9Gl1kpOh3eoBBb//KAEjQa3EEdNwijJwUj8USy5gc0lUPXgXFA3UAQVp0FsHzi3UQrD3vfL6GCbVVb0TeXiApo4X4SckNbwyz+g+x0ymtxul8ybA7Nh+Muw93NoPlQEqBM/wh1b4dBcEYsCm0uGRuFFybsZ9IyMTj77i/MxnVoFxlccQk5Qi4Z/4O7+Il4pNCpu7BrJ/D2uXSFTe0STEOpNm3Bf8suqOJtTxpxd9bdVqWTqVadoP/LKqlCpYESbMNYminurT/NAzuaU8u3OZKfXXcgr556FB9k6PdghpNSlzXhoOVqmMOWdEUF10LPQpCOqwosySvvYEnHXBMSJu+zy6VR1MVZPkxr+mrQCHppTX9w4PB+u+UDOqT2fyjll8KG8y90UtZxCT50NomaJqHJmnQg4ai30eVBG9p7dIIIvwF3bGz4WEKfN2Q1yrm95tf7z7W6UzzbuU3EPnVolOT43fiutVtYqEXlLs2DhDSLigFxzQlqJUGS3QcYR+byuCIoH3yhUmuo/r0e/a/h4938FnacrrhwFhUZCqI8boT5udImRIHNbXiFc2iOuRJVGcvFq2qg8gyH9sAxWaGx/y6vdkEUWLdG/d2sVyMLZqVXivNTL/ZCbTsOQlqEs2nuJ+wbHOw0RUFBQUFCojyLkNDI6Rvkxok0o6xOdgzwHJ4RgMtt4dHgCjyw+QkmlQ2gJ9jbw9oT2+LjrCPY2sOFkFrMGNeOB72TSUkygB2WVFvLLTYxo3YRvdl50qsV+OpzOtB4xfLLlHBq1iipL/ckMdjtsP5uHxWYnr6yKAymFaHo0h2XTHRkXIELHhmdhxgq4tE/Gf+99XoQW9wDo+4j8/+ZXYN61Iua4B0sB9t0kx1SItP1wbLG4WyouW3n3DIYBT0jhiAqmfCdBhCWZEt5aliMFX5MOsPEFEXGiesgEnLHvSxbHstvEWRA3SKbsrHkCMo9I4PKot6XVojxXMnPUGrmB6/MQFFyQrJFdH8GU72XKTd1WLbvNOfQwshsYvKGqtP4Pu/eDImwpNCqiAjy4vW9Tvv71otPjCWFeTO0RjVYjN/mBXgayS6soqyOKBnjq8XbTclvfphxILmT8Z47Mpk+3nmd6zxgeH5FA//ggZsx2Hc6dV2bCpqqTx6BSQWx/iOwK8SNEaOw0DSK7i6BaUQjb35TzJryLiKs/3gXXfwl2K8SPhMxD0v54edthx2lQkS9CbXmOtCeeXOG8jd2O9dJ+LnR8DI/Y69HZzeRWqogJCSBsx2toji+Ge3fD4puc3TYXt0lWVqebIHWPPFaaA9G9ZILV5XSYKm2O/jFyTsUNlKyptP2Sh9VzlkzFO/uL5PHUcGQhhLaVc3/n+3KeXy6C1QjPNZjLRagFEaAiu0FoO5mcl3sK8i+gTtkJakDj5vLnBFS7hhpvAL+Cwt8euw06TJOboDOrIbYvmCrkHuPMehFw1DpxoTQmqoWcYrMWrz9KyDm5QpzSTfvXPjyqbRhrT2SyYE8KswY1//3fV0FBQeEvhCLkNDKCvQ28el07ZvZpyvf7LmG12ZnUPZoWoV4UG83c991hJxEHJAj5rfWn+WByR54d04ogL7H4Tu4eTVyQJ+dzy/D31OPrriOnuJKlh5xblXaczWNU2zCmdI9i6YE0AjwNaNQql6M4x3UM551fzqACVCXpziJODSPfgB3viVumJAP6PgSl2VJslefKDVH7SSLUXNgGA3vCspvrj/a020Q0mbrU8ZhnkIwY/uVZ56IwsisMeAp+mCmC0eDn5AZr6lIJcd36umynUotr4Z49sip/erUUvjXtJnY7rHsC7t0L296A4z843mPPp5Lnc8dW+PFOmVY1ca64H2qEmqB4KEqFHe9K0RvWDm5ZDQtucDgfVCq5Mew0rWG3kcJVi7+HnlmDmnNNh3AW7btEcYWZ8Z0iaB/pR5ivc1HvqdegVkHrcB/uGdCccpOFwnITCaHemCw2NpzMpsLs+L2fvyeFBbd1x6BTU1in3fFy0qvc8QmKl9bGof+U8yh1L5iM4mRb/bBMclNrYPTbklV1aK5MbosbJCO9S7NktbT1NRDaStxjBRfF6WKphJZjsVeVoKrJp6rZl0egiCOWKtC6UdFxJskJt5FT5cEX27OYNag5F3MyaXniWTQnl8PMNRIa7qplKmUntJsgo8YLk+Xcm/AtzL/OuQVszDvS/vh5H4fgEtpGpmOp9fI5alx4vzxb/32yT8gx69xdO5kiu0qWTg1xg+V7MPpfcmzZSRDRScSo6sl12oOzxW140zI4+aOIvJfTbIhso6Cg0DgxlUG/hyH/rCxCfTfF4eSL6g4DnsQOqPWNLPPFbMRmV1FiVv++48drcPcTJ2LqXichx89Dz4AWwXzz60Vm9onFQ6+UKQoKCgoNoVwhGyFB3gaCvA10jxVrv1otf2TTCivIKql0+ZrEjBKKjGYeWnwEjUrF6gf6svV0Dm+sLajdxsug5cMpHRmUEFJv6s4/fjrBxK5RLLy9Bz7uWu7qH8enW89f/jbEh3pTWmmhZZi3TJK6nJA24Bkq4o1KAyvvl6yIGgKbwah/yQr3qdXQ7xEpgMrzXH8zKgoBu4gvicslC2fjC/WLwrQDIrTMXAcaLVz8FdIPSphp1lHHdnYbnPhB8jDsdglQvRzvJiJQ1RVxaji2BGL7SYGnMYjQNPhZWPukCDSDnpUMkCML5V9wSyn07twmN4GVxeIo8AwGN1/Xn1nhqsffU4+/p0w5sdntaKqt9hVmKzklley9WEBhuYnBLUO4s18cHaP9eXzpUaeWxc7R/nw4pSP3LjzkNI583u4UHhuRgL+HrkEx55zRk+CRXxBkz4elMx15Md3ugIUTHNPdBj0rLY11XTTphyD7JPS4S0TMuoHDsf3g9o2QeQJ7VDdUs4fJ480GQ4cp8O0YaDFcsnWwY/UIIqkilJmLzjOmXTivjW/L0dQixsbp0GxdBm0miGPlyG/k8DQbLC2OBRdg9WMw/SfJobq4XYSnyhLY85nz67ITRSCd/hOUpkPeBbAYXb2DcGyxjE6/3FE08ClI/MlRnDUbJC6flJ3Q5npYNAnGvCvXskrniTZUlYiQO/hZ+V7WRWsQQdng3fAxKSgoXNXYVRpQASdX1r8nSN0nIelj3v2fHNt/hamcEjywo8Lrj+pwCmklbWk2i7TTVnNthwi2nM5l0d5L3N4v7g96cwUFBYXGTyOch6hQg1qtqhVxAEorGl6hBwlR/faWbrxyXVsW7r3EngsFTs+XVVmYtfAw13QIR33ZAozdDksOpKJRq7h97gG8DFpeHteWlmHeeOo1dI7256MpnUjOK6dnXCAllRaMbpeN1g3vDKPfgt0fyVjiX551FnFA8nHWPSXFTf8npYjT/0ahU1kC3W+XANLwzpB13PV25zfLdKpPekBOooSe1hVx6nJ0ESSMcP1cyzEyhaYhji0WkcjdT8KMvcMhujdMmANJq8SFU0PuKWnD8gySVf/mQyRvRxFx/hKoVKpaEcdosrDxZDaD39nGEz8c4/W1p7jm41+5sWsUj10m4oCMEF93Iosx7ZzzU/LLTZjNNqb3isEVQV56VMC+fD32rW86RBy/GMm0qRFxDD6Sg3O5cAEyGvz7KfWnRiXvgP3fYM8/AyVpMkZbpZKpLCvuhcpC+f1fMh2WzEAz/zrauuezfao3L3kuIWj/u3Q2ZKK2mUVU6nWvtG/ViCSusFnFKVdD+n55r3Y3iugbPwx2fej6taWZ1cHlGvBp4vjsrjAbRVgd8Tq0ukayc27fBFpP+UzeYRJcOvINEYW73QYrZsmxewWLY8gVRSniZup1n1zXVGrJ2rpjCwTGN3w8CgoKVz02lVaGEhxb7HqD7ETntszGgqmcIrtk1/zuU6tqCGkjf2Oyjjk9HOxtoH98EJ9tO0+FqX4rv4KCgoKCoAg5fyHC/dxRNfD31l2nochoZuac/UQHerDkQKrL7SrMVpIySvhkamf8PBzLMD7uWl67vh3lJgvpRZW8tf40c3Zd5NoO4Sy5uxeDEoJ5dXUSL6xI5Na+saQVGnFv3lfyZSYtkFaN0f+CRRMloyK6V+3443rknZH/thgmLhuPAAkgdYXBW/IqVj8GhSlic24I7ybiAlKpJdOjNKvhba1maGjGhN6r/sp7XSqKRLgpzZTPrdFB62th/dNwYin1MjEOzYNy1wHWCn8dsooreeD7w04tiS1CvDmZWeKUk1OXVccyGdbaOSepf3wQc3cnM7x1GDf1iEZbR3XtGuPH8pltSHDLZ3C0FlXcABEJAQKaitOmhqjuIm5ejn9Tcb6YK1x/kGOLUYW2QTVvHDRpDz3ukUBPV8XKNR+g3/EmvgtHod39IV573iFq8VC8Ti0WQTTzCFDdztgQCaPEeVOXLrdA5lG4uFWyrOq2WV1ObpKcs35RIjw1REzv6sD1JnI8vR+QDK/Y3uJCGvaStAEsmSETuiqKJRy62RCwuv751WKzwJAXYNY+eOi4tIeFtgGtEuapoNCYMel9RQS2mhreqNxFu+bVjqmcfKTt0+ePEnK8w6QVN2Vnvaeu6xhBkdHM/D3Jf8x7KygoKPwFUFqr/kIEeOiZ0DmSpQfrj+O+pXcsa49nYrdDYbmZSnPDK+BphUbC/dx48do2GLSS0WK22thyKpvoQM/a7c7nlvPW+tOsPp7JI8Na4OWmJb+8Cg+dhqV39cRgSYaU3ZIt0WGy2HpqVvivdNMDsrpt8JHVbpMRhjwnI4IvZ+SbUFkmq+Mqteu8iYRRUvgVp0F5towbvrDtyi0Naq1M9FGpXIwYVkHzYVJIuqLpANj+lkzL0rrB8FfkvUsypEi+fAqQ2XhlR4LCX4L1idn1fpX6twjmUkHD7T5VFpuTOOvnoWNM+ybcOucAKQVGnhndkus7R1BcYSHcW08zVRqadbejvlR9YxzTRwLBN70s2S9db4fmg0V4cPMVsebsLyI61uDh/9vOlRob/NY34Nb1sPW1+tuFtZf3dCHYqksz5Frw6/tQfAnu3glHvxf3itM+OkibYY24q1KJK6bpQCi8IDlXGr0IvZVFro83pLVosqsfERdQRGdpH6uLRgd9HxX3T1Bzcfgsu9Vx7ns3gRvniEtH6y7bVxaKeBUYJy46jd71dU2jl8+g1SvTqRQU/kKUVZpRWc3YDV6oNLqGnTe+UX/ugf0emI3k26uFHMMfJOSoVCJoJ++CHvc6ZQKG+LgxsEUwn249z5Tu0Xi7KaK3goKCwuUoQs5fCLVaxX2Dm9M0yJOvdlyg0Ggm1MfAA0Pi6Rjlx8mMEq7vHIGnXkO4rxsZxa7zdOJDvYny9+DNdUlM6BqFRq1Cq1ExvE0YKheWn8SMEl5Ymchd/ePo0TSA+XtSuK+jFvWCaxwhv006ws4P6hysVoqhmhsfjV7aGcI7iavGOwyyTkC7ifLHPu0ATF4Ih+ZLURfYHHreI2LL8aXiajFXiOsnfris9JuMsuofXh1CWvcmq8stElIa0gpykup/E9pcD5nHZDT57o+l190zWN4zuKUc+8EAMDq3p+HuL1N2Pu8tX1sqYc1j4ko6shD6P1o/yyO2n2McucJflszi+g4XbzctHaL8GnxNiLeBsioLapWIPnf2i+NiXjnPX9Oa9zee4bpPd9EtNoAIPzdeH+iN7pvhzu1QKTvFtn7jXMg9LefF7o8kiBjkJnrcx9WOtuopWwUXocfdDX8QvxgwVjvIzBUioIS2rZ8P0Xqc69HbnsFyfi2unvRi8JFpVDfOkVbK06vl+tD5FsmjqSqDG+dJi1hIK7nZnzvWITbFD4OuM+HX91y/V0AzOR9tVvjlHzD+SxGYjy4SZ11sf+j/GBj8wJgjeTgH5zjvpzQTFt4IE74GlR3mjHLO7aooFAfPjrfrH0Pfh+U4FBQU/lJUVpbjd3EbqlYjZEDDYRdZX0EtGuf5byqnwC6LXd5/pIYS1l4cl5lHIKKL01PXd4pg+9lcvvn1Ig8NbfEHHoSCgoJC40QRcv5ClFWZScosYff5PP4xphUeei3FFWZWHkkn2MtAodGEj7uOM1ml3D8knqeX18+SaR7ihYdeAyo747tE8szy47VTsLwNWhbd0YNmwZ6cz3UUi2qVBC0XGM1czDdSaqzEO3GF80htcwW4+ThW/k+vddz4hHeCIc9LUPC+r8Qp4NUEwjtIkWcySrbNT7OkKOp+B3gESfG09BZHJo5GJ0XowGekeNV7iljzzdD6rpqDcyCim4QQrnvK2V3T8hrofb+0UfhGQPe7oPud8nmadIQvB8j0qfFfw8Fv4fQaeV3CGAlntpgkD2fvZxLkByJATf5OxJy6k7w0OhjxquTpKPwlySyqYH9KAR0i/QCH46RlmDeDWoZQUmmmVRNvkjLrj6B/aGg8Uf7ufDy1M/uTC3hy2THuHtCM19ee4ptbulJaYSbIy0CAuwr9wbccIo5fDER1E/HywlbJYrKZ6wsN2Ynw072S+/LDTHmsohBQiWCZe6r+B+rzoEPo8I0QkTWyhwT3Wqvg/BYJAw5rB/u/rv/6TjeJmGm3i4tu4jzY+E+5kW86AIa+KEJuYYqISz4R8jlykyA4Ab6f4ewYOrcRWo2DTjMkNLnG3RYQJ/ve84lj4l3T/tJeGdtXpsa5+YLOTY55xzsw6B+yvSuqSkDvAwXnYfQ7UFEgQlXqPhGxhr8M13woDsTiVPCNFEEsrL048xQUFP5SeFlL0LYZLdeCTjMkry9ppWODsPYiHGs9/ncH+Z9iKidfHYC3HjSXhyb+nviES8bYxW31hJxALwPDWofx5fYLTO8ZQ6CX4Y87DgUFBYVGiCLkNAKMJgt5ZSZySysxaDUEeukJ83Gr546psth4YtkxSios7DjnnLkysk0YyXlGXlsjhdndA+J4bmwrPt96gdyyKtQqGNY6lKndo3l+RSIvjmvNI0uOYqkzLae0ysItc/az+M6e/Gv9aTpH+9M8xIsKs5VWoR742UsoqzJCvAcehy7LtDj5E7SfDJtelK+PLITrPpXsivhh4pip6yRY9aA4dHrdJ21Wg5+VcOSMI9ByNKx8CLrMcIg4KrWMHT+2WMYjg7RUeYa4aI2qZu9nEtJ6zQdSKJorpN0qaSV8PVTaJFQqyQBx95NxzaPeFJt03lkJdG0/CW6YLX3eWjdY/wxc2g1eodD5Zvm38j5pA/EKcfSEVxZL0Tr4H1CQLO/rEynvaakUh45S/DU6iowm8stMtaKpu07DPQsOciKjhPcndSQ20IPkfCMBnnpevb4t077eS5XFyoeTO/H9/lQ2nMzGarMT6Knn4aEt6BDpx21zD9ROo+vdLBCVSlqsdGo1+5ML6Rzth7aqAvXFrSJMjHpLfpfPbxaR4rrPIKA5zB3j+qDLsuX8SRgt51rcAPn9nThPzqVTq0QI8Q6DAU9Jfk5NOKV/M4gbCOc2VIcm26HtDdD3EdlHbF9xzNUlKB72femY7lbjFAJoP1EE3cTlju0NPnDtR3I+l2ZKbk1d7Hb4+QHoMhPu2SVOGZ0boJa8qqSfZbsOUyTTZ8l0hztPpYKe90mOUHGqjB83Ol87a+l8s7gF930pn98rTJxAnWfAygfgl+fkezblexHD8k6LUH1mnRQoiutOQeEvhc5ukb/dp36WyZQDnxGh21Qu15KCi/D9NGnnJvQ393dVYS6nQOWPj+4PFHFArsFhHSD5V7nX0jmPaR/XMZytp3P4aPM5/nltmz/2WBQUFBQaGYqQc5VTWF7FvN0pfLzlXO0I4hBvA19M70K7CF+0GkdedV6ZiZKK+qGb/h46fD30LNp3qfaxz7ddoEuMP0+OSiA6wBMfdy2/JGZRWmXh6dEtySyq4svpXTmaWsSnW+W9O0f7Mbl7NPllJp4Y0ZKHFh8hr6yKueObEHroS7xOLydQpabJsFexewY7RwVnHpW2pLiB4hCw28QJMO4T2PN5/ek4IAVYu4kifCyaJK4B7zBxAIx8zbmVImGUOGjOrHc85h4AZVcINC7NlPyMeddKGOn4r+T/62K3y4Sa6z4VZ4C5QrIuQI75wGw5nu53imBTu+8s2PYmtBwrRW3uGfnMGYdg6D9B5yHF68YXxcmQdhC8UmH3J5IZEtENet8njiKtsgrVGMgsruCZ5cfZctqRgdQh0pcnRrbkwe8P8/Kqk7w7sQNLD6YRF+zJm+tOkVsqLU53LzjIDZ0j+XRaZ+x2O5H+HpzKLGH+nhQeGRbPE8uOo1LBPQObsT4xi8eHJ2C22RjbvglZJVWY0WH3CEI16BnY8jpkn3Ac2LElMGNl/WymumQcljHexjyZJOfmLyJl04EyHcpuE0ea1SpBwJFdRfBoOkAKldw67YmlmeJi848TZ9uFrfLekV3FfefmC+0mQefpEpJ8fqO8LravCEl1RRwQJ8yPd8pnuLyVsQa7TSbJtZ8kYnGn6eIoqiyStgZzBbS5Tq4jTq+zS6vZ2Pclv6qiUM7zy8WiiM4Q3lFGt9f9nFtecwQj73xfMoFS94pwU5gi39d2E6V1VEFB4S+FylQmrZ5Fl2QRZtM/q5+4PFuvgcWkqxlTOQX4/nFBx3WJ6CLOypSdMtGvDj5uOq5pH86CPSnM7BNLTJ2cRgUFBYW/O4qQc5Wz/Wwe72086/RYTmkVU7/ay/qH+xMd4LDsWqyuA3OHtApl7YnMeo8fTCnkYEohrZp48/r1bWkT7ssTPxwjv9wR2DkoIYT3JnXkWFoxTXzdeHv9aR4Y0px//pxIUmYp62Y2pcWaGyXMtxr9modkhO+5jc5vuPIByePo/4Ss8qu10n6x4l7HNp7BkmtTI3Sk/CotEJXF4ho4sQzaTpBWpborN22uh1UPO79f3hlx9dQVd+oS3kVWyyuLod0E2XdDHJovRWL8CPCLlRu1qhI4NFeKxm1vun7dqVWykh/bD4pS4dwm+VeXpPYyNn3etVJ8egbBhc1w7DspXmN6N3xcClcFpZVmXv75pJOIA3A0rZg3151i1qDmvLI6ibsWHOTaDhEMaxXKh5vO1W5XabaxcO8lFu4VsfWtCe3JKq6goLyK5iHeNPFx44lRCfx4KJ3lh9PZeDKHD6d0ZNaiw1wqMBLqY2DtDU8TkLLGWcSpPcAMyW9qaLpTWFuH2DjoWewGb1RrH5cgZKhuf3xBBJHVj4gg2esBaSOsK+L0vBciu8Hh+eKOa9IBZvwkmTxnN0gWgl8cdL9dwr/PrhPRCKode/90fXyWKkjZdeVzQecu7ZRVpeLQ0XvB1O/FdXduo8OZ44pD82HgUyIyTZgtrqOj3zmmZXW9FTa/6vq1icth6hIRhAKaSqumzQIDnpDrSs97HOLvf4vNJqKY3SY/T53i2lNQ+G8oqTBTXCEOPV93HT7u//9AGJXWAKYKuc7Vpa6IE9gc1I1wMcZkJM8ejvefIeS4+8n36cz6ekIOwKh2YWxMyub1taf4/KYu9V+voKCg8DdFGT9+FZNbWsn7l4k4NVSYrWw97TzSMtzPHZ2m/h9dN62a0kqHU6dH0wDuHdiM2/s1JTbQg9JKC54GHQ8vPuIk4gBsOZ3D/ouF9GgawIs/nySvrIoWoZLn0a95AOEpK51EHEBWpQsuYO85y/nxdjdKS8OKe6D1deLSyT9fPRJcJYXi8FekwEs/KCO8202U4gxE3LGapBXk+BJoMdKxb7XOOZMHpJiK7ScFz+Wo1NDrHse4co9A58k9l2MskKLs4Gw5/qU3O1bjA5tLW0ZDlOWIk+jy71MNB+eIPXvqYpnuFdwS+j0mDqFNL0PpFSYIKVwV5JeZWJvo2v2VmFFCbJAnKpUINksOpJJa2MBo72rKKi10iPRDr9UQ5KVn0R3dCfd1Z1rPaBbf2ZNvbunKk8uO1068yi6pokzn37AYeXRxwwHG7v7SLtj1Vjm/Zg9HVZwmDpSpS2WiVPvJIlbu/0qEibu2Q+dpch7W0HoceARI1s75zZB/Dk6tlt/fxTfB1tfh0i4IaSEtB6ZyEYJaV7vgDN7OAcKXU5wm4kzcQNfPd71Vri8jXpPCylQm1wW9lziLSho4v9Uacb/lnILld8DsEdLGGdNX9gXgEXzl60P+Oeh6m4wkt1Vfa3e8Az1nQWCzhl/371CSIcHrs4fDl/1h/dMiODXUOqqgoNAgNpuds9ml3LfoEP3/tYX+/9rC/d8d5mx2KTbbb59TFpNJrn+VxdIG7RfjesN+j8h1qLFhLiff7o33HzWx6nIiu0HOSXEkX4ZBq2FStyjWnchiz4UGWl8VFBQU/oYojpyrGLPVTnK+i5ajak6kFzt9HerjxgOD43lnwxmnx4+mFXNdp3Au5pXzynVtOZhSyPazubjrNMzs05QWYV4cTy+uDTW+nKUHU+nVLACQwONCo6xejYzT45P0o+uD2/IqquGvwsx1cHGrFEIhrWDOaHm+LEvC7fxjpP0osLk4aGoydEBaFA7Pg5t+hGZDReyxmMTRUlUif/Q73SQ5FJXFEppXkuF4fUgreW78V1KEpu2XxwPiYODT4hiI6iGPlWRJO0jPe6Uwyj8n722pEoFo4NOw7mk55hpKMyUT5/b645Wd0HvA0plSsA55Hja95Px8YLy4k767SfJxavAJh2s/lgBF70bWX/83o7TKcsV62mqz88VNXbDZITG9GC+DlhBvAznVrVWX06qJD4dTC7l3YDPyykysOJrBgeQC/Dx0TO8Rgx24mOd8bTibV0m0uYFR5he2iJBzeSCwXzRM+FZyaA7MlrHbQS1E3Lm4Q26uF0+Tlqcazm+GFiMklFhVZy2gwxRYMsP5fdtcL4HfMX3ESaLzkHaq4ARH0HBZjrjWStIkPyfPtXhNTG8ZT97vURFILNVi2LnN4mLzDpNWqj2fiWtv7AdgKhFxZuz7EN3L5Sh0etwjgtOJOlO3jPkyUr3/EzBrn2PKV0P4REhr1qXdjsdsVmzFaRj923DFdJySTGlps1kkxN07TELQL9/m+2nihKrhwGw4sRzu3CLXNAUFhf83qYVGxn+6i9Iqx33PtjO5HEopZPWDfYkO+I0WnrIsVKdXi4B7dLFcRyvyxbVorhC3b3QvKC/ArrLzJ8khvx+VJRTYPEn4s7pCQ1qL6H56nbSqX0af5kFsSMrmhRWJrH6gr1OsgIKCgsLfFeVKeBWj16hpFtxwCdApytlpUl5lZlS7JnxxUxfahPvg46ala4w/9wxsRodIP96+sQPP/nSCT7ee50R6CfuTC3lhZSLrjmfVZnW4wmiyUrdrK6h6coDZxpWzH85toMpsliIprK1z+5FGL8LM8R8kMDSyi+tRxWU5kj0x5HlALStbNROetr8lBdSNc0XA6fuI82sri2WKzop7ofkQGQE+cZ6INTvekeDkiiIpHrvMkAlUi2+SMNSTP4mIMuwVGX3cpIPkZLj51j/GsxtcuwT0XtKOFdJGiuOtr4PeG7rd7rzdoKfF4VNXxAH5TNv/BSpEzFG4avFx03KlwR5eBi3J+dIC1bdFEAeTC3hkmOtxqtd1DEenUdEs0BOLzc4t3+5j7q5kEjNK2Hkun8d+OObyfN2UbKKy2aiGD6I0Q1oXb1kFNy2XUN5+j8Ky2+GzXiIk9H9cMpwW3iC/62fWO4s4NZxZL7+f7aszZ/Re0rZlrePoC20j55paI6LkgKegyy0iSvpGyrlbWSoCj1cIBLcS4cQV3mHiEPKPlXDk1Q+LaLTsdrBboNlgEYENPjLZrvlQ2WdliZybqx6U9jCDj2OfWjdxEbWbIEKKysUPcPdHIixp9CJGuULnIUHGqXucp98B6opCNIe/xVKcUf91VotMvPpmKHzeF74cCJ/2lCD4iiLnbbOOOYs4NVQWwY73pHBUUFD4f2G22li8P9VJxKmhtMrC4n2pmBtoVa9BjVXcwjYz9H9M2jdPrZWclxPLZMKlRxD4NAH3wD/ok/xx2CtKKLC54/NnOXI0WkdWjqX+9UytUnFLr1jOZJcyd3eKix0oKCgo/P1QhJyrmCBvA4+PSHD5nLdBS9/4IACySyo5nlZEodHM62uTeG1tEsPbhPHF9C7c1rcpBeUmdGoVm5KyySyurLevFUczSAjzbvA4gr0NVJjlhqe0yoJeo6ZDpC8/JBkpaD2jwdfltLqZD056kTLoY2waN+fWopSd0GwImI1y85O6r+FvxMmf5L8BTaHjJCnaQJwz296ElfdDTqKMNx/5Jrj5yfPnNknrU1kObH2jWqSZAWsek7HKPe+R54a/KgXhha2O98w8CmufEFfPxW2wcpYUtOM+kQBXkAJVpYJ9X0gmkHcTx+v7PAAzVogDojgFetwFt20Q4arjTfLeE2aLuOTdRHKAQlrVLzYv7ZZi7YeZ1RkmShvF1Uigl55xHcNdPtc52g8fdx3llWbWnchi0hd7+GDzOY6lFfPepI60CZefd6iPgWfHtGLWoObY7dA6woe315+u55QrN1nwNNQ3Uy47lk9Ox3tdi42hbURw2P2RtDQtmgTfTYafH5QR3zYL/Pqu/P7lnxNHSsJo51G6l3NkkYgbTTqKw6euO8c3Ulol546V86dJR1h0IyyaKOfrV4Ng8ysw+i2I7S2ZOktvlmMb+560OtYQ3VMmb2UnQtYJOX9rriVWk7jufvkHjHlXcrUSl0ur48YXJKg8pBVM+0GO94av5Fg6TIEb54gAteEFCG0r2/S8V1bWJ86DyYtEePIMlglUg58Vcaguai2MeQf2fS1tnHUde2oteIfivuFJND/MlAD0uhRfklysutdFU5n8TDIOOx6z2eTYG+LUzw1nHykoKGC328kvqyK3tAqL1UZppZmtl+WZ1WXrmVyndnSX+1Rpoevt8jc5JwmCmoMxVyZphrWD0NZQmoE9pDVqfePLsiqrMGKya/+csOMaorqDyejIJruMuGAvhrUO5e31p0kvUsRrBQUFBaW16iqnR9MAnh/bmn+tP02F2QpAdIAHn07rTISfO2mFRubvTmFC50hWH89kU5Lk5ny57TzxIR24d6Gs4r52fVtWHHGxKgwUGc2ooHY08uXcMyCOwursHJsdDl8q5KlRLflw0znO+vWmS3hXtBkHnF5jih3MMXtzNpwroF9UIJFu5RL+6REAF7bBwblww9fStuQfc+WMGVv1ype1Cgy+UnwN+odjzHhViQSZGgtElBnzjhRRGp1sP/xV2PCco50EpPUipi9E9oDTq+sXQh4BMOwl+HaUCCk1JP4oroXbNsgxe4bI+9htcMtqWU3SeYoDadNLElpst0tx3WuWjFvGJgGvPz8kxx4/XNxEkV1FLOr9gBTSG1+QwrYsV/abdgDu2iauBIWrCi+DjqdGtcJqg5+PZdTqbb2aBfLitW24dfZ+Xr6+LTPn7K99zaJ9l9hxLpfJ3aJ5elRLbHZ4f+NZXlkt4cEbHu7PjnP1M2Psdmmr7Ns8kF/POfICqiw27lqVz6Jp6/E9+Anq06tA54G98wxUEV1g2W3iTDu1WlaRXZFxWKZNTV0iv6P2K6xK263y+zn4WRE6vMPE5WKplDbFTS+JK67PQ+Jyu7w9KWmFiLMegXDzKlj9qAQp37peBFmNTlZpM4+K0DrkeXGoNXTc3k3kHPEOE8GohpRdcr2Z8C1kHoZRb8qxLJnhOLcvbpOcII9ACUquKhWRNn44uHlLFkara2D0O9IClnUcvMMlmD1lj7QCbH5Zrk01uVsDnoSj3wOgSt0jTiLvMMdxJf7UsJNm88viAvQIkOOoyQlzhdYAja9xQ0HhTyGruJINJ7OYvycFk8XGmPbhTO0RRYBnw7k1AZ569Norr3OqKgoAu4jgGQelpbOGlF1y7o7/WjIAGyH5FXLt/9McOSDfs+AESFol114X17VJ3aI4mFLIU8uOMe/W7qhcOSkVFBQU/iYoQs5Vjp+Hnpt6RjO8TSgF5Sb0GjUBnnpCfNyoNFk5lVmCh17DiYxivt2VXPu6HnGBV1xxupx1iVm8M7ED7244w87q4tDHTcttfZtitdnpFx9Mt9hs9icX8s6GM3wytRNj2odhNHhQOPYbfPKPYjg2H7tKS26rmzhsiuKL/cXMH20gbFW1K6aGNuNlJX7Z7RIEGDdYMnP2fOr64BJGS17OuielUOw4VVbUm/aXtiazUfIz3P2kuFx2m+O1KpXcTN39KyT/KgVsTB+ZcJN+UESk5F/rv2enGbDzA2cRp4bt/5Led41ewluTVgF2CSqO7Qdad1g81Tnro7JYwpG1HpIbsv8rEXGaD5XsnHnXSFBrDRFdJNtn1UNSxPlGiXB05DtpP9E0zpvDvzKhPm68en1bHh7WgpIKM+46DUdSi9iQmE2Ir4HT2aX1XpNaUMG/1p9GrYKPp3bm0CWHoHgla/+nW8/x3R098XW/wLrEbKw2Owatmt7NQ9hX7Mc20wyGDrodH3c9HS1H0BaniZvMK0yyX1yhUou4eWC2iCxtrocWo8Qt44r4YbDsVug4TUaHG7xhzHsSBh4YJ9OzQlpJllVDGTOH5kqgcNJKEWc1Ovn/X99zbKPWQKtxENqu4cBwkPO5wyT4Zlj950xlsO11yQlaeb8ISOM+luvHobkicFXkw7a3HK+x26tbyDKh8wxx1Gx7U44jsgsUp4so3Xa8CD897xEB2SNQBLAza0X4rSHxR4gbIP9vtciY94bIO+sQeVQq6HKztFy5ovMtIigrKCg4kV1Syd0LDnIktaj2sU+2nGPL6RyeGJHgJITX5c5+cXi5cD3WUFZSiOe2NyGkrWS71BVxajAWwN7PpD27EZJVIfcY/n+mkANyb3XwW8g+KU7Sy/DQa7m9XxxvrjvFgj0pTO8V++cen4KCgsJVhCLkNAL0Wg2R/h5E+ns4PV5QXkVWSRXvbTzLp9M6147QBNBpVE6F4PazeQxvE8p3+1w7X4a1DuW1NUlM6RbNUyNbUmWxodWoWXogFXe9hg82nWV4mzDuGdCMtMIKMosr6B4bgEqlosDuhjF8KJrQfuSUmnjsx1NcyEtj9YwYwn4cJYJFXRKXS8hqZDc4NA/iBkkrRsIoOL3WeVs3X8mUWXk/DHpGJkOU5UBhiqxuZx2TVqrg1pC2V26qpv0gbVKH54uAsuw2ed30n+TrXZ9I0dZ6nOy/7ip5DZFdJZunIfLOyir+6TWOx7KOy2rSjXMaDmzd/aEExU5eJJlACWPgu0mOSTc1pB+U0c9jP5Bw2f6PASo4ugh63u16EpfC/xxvNx3ebrLSm1ZoJKWgnHM5ZdzQJZLMIkdbY6+4QB4e1gJPgwabHSxWm9OkFJUKTmeV0j8+iG1n6rtyKs02Co0mgrwMfDatM1abnabBnuxPLuDuhYew22H5cTVPD4ulU6AXpB6XjKbglmL7P7+5/sEnjJaAzqQV8vWpVTBlMZxd7yzEgginFhMUXZL9RnSFwc+JNX7qYkd2lkcQlKQ3/A2rKBShcs9n4o7T6OU8qkGthfFfQvJOKE2X5+vm8NTFJ0LawmxW18+nH5L2srwz8u/Mehj5urR4dpgCqx9z/bqsY9JStfdzcfWcXS/B6T4R0HywHNvKWQ73kt4LrnlfrlF1MdRpX9VoIayDuKNcEdjMebR4QDMJhD4013m74FYiMinCroJCPY6mFjmJODWczCjBZodbescwZ5fzeTqzdyxtInzqvaYuGlMpqgub4dpP4Mi8hjc8t0kcwY0Nu51sk1zDA9z/ZCEnqLlcb5NWuBRyADpG+TG0VSivrE6ie9PAK0YDKCgoKPyVUYScRozJauPDTSIYZJdUEhfkyYXqSTYHUwp5elQrfjwsRdTGk9n8cE9vtpzKJavEOSdnQItgsMMb49vz9a8XeXbFCSrNNmIDPXh4WAvahvtyPrcMPw8dWrUKPw8953LK2Hwqlzm7kskqqaRZsCfvTuzIsmO5JDTxxmyzEVxyor6IU8OhuTLRypgHG56XVewBT0v+zdHvRXCJGygF1rpnpDDa+KIUVTWEtpE2iRM/SkbO1jdkqo1KJS6fCd+KABTVHbrMlDBSUxn0uV8KxOM/iODSckz9oOXfyqKxVsl7gQQZd5wmIa6FyXBpP4R3dh1OWpYjx7Bwgoxj94sSJ8/hBfW3PbZExq/XCEoxfaRN5UoB0wpXFR0i/agwWYkL8qwNCZ/cLYrpvWJ4d8MZtpzKwWaHDpG+PDO6FQtu64HJasNoshAb6MkTI1pyKGWPUyinRq3ixdHxJLgVERZnotSci09oFPllVfySmI3dDkPjfXmxnwehx95Dc/yoTIUb/oq0QMX0EdfN5ROu2o6HlQ84vraapdVp3Kcy7encJnGytZ8MnoGS5QIQ1h5GvQEFydKmeGQhTJwvN+MFFyRMuCECm0loclALuRbseBtCWkorIUg7YuJP4tKxVIib79wGOd9CWsl14vgSyaho0g7ObbnyD+Ty83rzK3D9FyLCuHLf1VCY7HAEHZjtyLvZ/5WIrbF9HbkOpjL48a5qEewXh8DT/kbnfbabIJ/XlTA18BnnnCDPIGkt6zAF9n8t79FhCkR2B1/X2UwKCn9nTBYbiw803LL9zPLjLL+nF1O6x7D1dA4qFQxsEUKIjwE/jyv/jTWozCLY6t3lOhnWTv6OewTKPcXhBVCWXZ0d1ghbfyyVZFt9cFdbcdf+ycevUosrJ+lncWp7uVhoA6b3jOFMdgl3LzjIz/f3vaKDSkFBQeGvinLla8RYbPba8cWpBUYeGBLPQ4uPAJBXZqLSbKVLjD8HUwoJ83UjvdDIa+PbcTilkM2nc3DXabimQzjueg1hfm48/sMxp9Wr5HwjD35/hDduaMfcXcmUVFj414T2hPu5czy9mPl7HCtZ53PLWXowDY1axX0Dm/PKmiQMxecbPviKQilG5oxxPFaaKTk6I9+QVgzvcPh+sgge2950FnFAwk93fQJdb5Ug1Rrsdik8c0/JdJ6UnTD/Oue8jw5TJIvm8z5SSA18St6jptDLPSVTqtJdiDEgxeuml6S4cvOV4qowRYrQPg9LFk7yr+IMurjVsV+dO6jV0uZSnCbjnoNbyXHufN855M9slOK2hpSdEr56pbwMhauCgvIqKs02Qn3cGNcxgpJKM6ezShndNoxb+sQy45t9TqPHj6YVc9M3e1l2T2/umn+AzOIqnhrVkqOphXwyrTObTuWw/2IBQV563h/TBN8jX6BZOJsIsxG8QrAOeJokv4F0jvYnvbCCV3uYCV0xSZxcba8V4UHnDrEDxM01eZGEBuefkwPwCpXx2TX5LjUUJsu51WyIuFcKL8l5UVkMo9+WsdcFF6GyTAqWC1skN2frazDoWcmbMXiL485VW1TvB6X9YMgLsPxOef/e98H+b+S8jO0rLY4g5+LEBSKA7PpIzjmvEOg0Ha77Qtomo3s0/EMJblk/i8tULg4etUbauuq2N9bQcZq0TCatFGFm8HPyWdc9I+LP2ifENVT33LVZxfVUE4Dc91HwiXLer1+UuASX3iyTu0BaR4c8D1Hd6h+HZ5D8i+omAchaRdBV+Ptgs9nJLqmk0GhGq1bh76kn2NtwxdforjCiWqtRodWoSQjz+LccHVWleehPrpSJfeV5shCkNYjAW5Ip2Vaj/yVCRGkmaNz/3/u+aqgsIcvuT4ChAXfjH014ZxHzE3+SdlgX6LVqHhzSgmd/OsHjS4/y6bTOSl6OgoLC3w5FyGnEqFUqfN11FFeY6RDlh9lq443x7XhnwxlyS6t4Zc1JPp/WlQt5ZdjtcDanjA6RfsQEevDwsHiCvdw4fKmQY6lFtGriTYcoPy4VGCkod14h/mTLOZ4d3RqtRsUX2y9wa59YFu6tP/5x6YFUlt7di6SsEp4d3Qp7nuv+c0DGEeeedn7s0m6Yf70UlbdtkIwNnbusdm1pwJ4c0bnhzA9jvjiCVj9S/7kz66DFSBF39n0pq2lTl8hUHLVWCsjoXjKG+fJ8j07TRehpd6O8x6aXHM+lH5Jg15FvyvGHtpHsjJ8fkGK6xz1groJVDzuKNxCXzQ1fi3iTVp2dEZxQv/A8NFdyS9y8HZ+xJFPcRnovcR95hYHeuQ1P4c+hwmwlMb2YF1YmkphRgl6j5okRCXSPC+CDTWdZfk8vfj2X5yTi1GC22vl48zkeH96Slccy6N0skABPPauOZlBcaWZwqxD6Rmjw2vQkmrN1WvrKctCsfphWw14ltNvNzOzgjt9P0+H6z2QiU/YJx7bBLcWptvllcbuEtpXfS1OZiBieQVKc1MVul5vqdjfCwdkiYqx+WNqUet4D5kr5Pa8slKBvkN/h4JYwbSkcnA/Xfiy5NzVTnTyDJR8r77RcB/ReIgoFNBXnz03Lq8+POjfmvWbJ9kumO0TZwmT5LBmHHe1eve6D3R87fwaNTlozN/6z/g9NpYLT6yQT6NgS5+c6ThXX0FeDHI8dmiufd/yX8P0UOWctlTLNzidCzsnjS+XcbXM9DHtRAsovb4fU6OUac+c2+axWswhTXqHObVWXo9Yq8yYV/laUVZrZdiaP51ecIL/6/qRZsCfvT+pI63BfNOr6Bbxeq+amHjGsO5FV7zkQZ2Sg15WFIFfoK/NRbX0NbvxWWqL3fA6H5jg2uLRb/o37FEJaYjf4NL4o8spisu0B+Ov/R1MytXqZVnhmvQhmDbSSh/u5c8+AZry78QxfbL/A3QOa/ckHqqCgoPC/RbkdbISUVZlJziunrMrC7Ju7MqV7FHqNGn8PPcsOpvHkyAS+vaUbc2d2Z+f5PH45mUWvZoGYLDZmztnPE8uO8eyPiXy78yItw3worjTz0PdHuJhbxkvXtuHWPrFO72e3g7e7ljvnH+RAcgE5pVXYXPx9r7LYMFZZ0KrVTPh8N4Xe8VLYuKL/41IU6r0k42HI8zL5xSNA/qXsEvGm3Y0NZ2KAtGTUHdVbl2aDXbcsgezTXme16ej3MunmaHV2zbY3JED01l9kRd4/VnJzJswW0ccOtLkO9n7hev/b3pDMn90fw5rHYez70HyYFHa/POMs4tQcz4r7ZCpOTetU30fgwLfy/yqVrO43Heh4TVmO5Hp83kcm/vx4F3zcVVYCqy5zVij8KZzOKmXiF7tJzJCWQpPVxrsbz2C3Q5smPqhVsKuBgE2AAymFtI3woWmQJ7fNOcDLP59ErVZx/+B4OkX60da3Cn1dEacOmu1v4ld6Fh9roeTdbHvLWcQBESB/ukdujs9thhPLYOkM2dYnXNxkrvCPlQtBt7tE1LHbxc1yfqs44tY9LSHgdduoDi+AojTodqsIN6PfgXv2wMy10npVnCnn3TUfi4DUaiyggpZjpRBa/ajzuR/aVkQbV5O0Tq0SIWjb63L9mDAbonrIcbebAFOWwME50upVF4OPCCOVReKaienreE6tgbY3OAu1NWQnSp5XwmiZZuUTKYLUzvfFDdjvURFtu9wiI90byrRSq8E3AsI7itPGP+bKIo6Cwt+QU1mlzFp0qFbEAXEBT/pyD+mF9Sdt1tAyzJsRbULrPd4s2IvxnSNdCkBXoshokuue1QQrHwZzubOIU5etr2PXe6H28P233uOqoEocOf5u/0MJKrq3XB9PLLviZt2aBnBdx3DeWneKHWf//wM+FBQUFP4KKI6cRkZWcSWvrUli1bEMbHbQqlVc2zGccD830gsr2J9SyP6UQka2DcPboCXY28Cd/ZtRVmXhs23nmdknlv7xwZRUmvHQa5n2zV6s1arM+dxytp/N45FhLRjZNqx2JWt6zxheXZ2E1WbHbLXj5dbwr42fp573Np4lzNfAxUILUVOWoFn9IKoal4neC/o+BE06wvktIugcXiDCjV8MjHlX2gs2viCFkWeQFEEqlevcmspiWcW+PIxVrZEVnZSdrg/UVC7vY/CWiTMgDoIWI6WwK8mQqTNbX5dQ5PFfQ+EFyQEx+MpUHlN5/ZDiGioKQV093rQoRTI82t0oq/Q134vLqSoRh8E1H0r71NkN0voS2U1avy5sE+dNZZGIX/nnJDi6LjYr/HQX3LtXHD0KfxpFRhOvrj5ZK3Jq1Comd4tiYEIIJouND6d0JLXQSKhPw6vA/h46yk1Wvt2ZDEh+1bDWoby6Jom0QiM/Dy1u+ACqStDZTWBXyxSmhlxsmUdg7LsyrttsFCeab4z8XjYbJNOjfn3PkaET3Quu/VCcadmJIrR0nFrd2hQqgceXdsm2HoEyTe7idnGyYJe2xl6zJKw384g44ExlIoLcul7cLLNHyLnc814ReI//IL+/Bm/J9qkJQc891fDnzzwiYs6mlyRMfeJ8cQCZyiXcvMYNVINKJdlbBm8RbNL2Qp8HYNDTkm8R0ExaqRri+BK4/ksp5uaMdghMpVnizOv3mLRguv3/2zYUFBScKa4w86/1p10+ZzRZWXsii7sacGIEeRt45bq2TOkezdzdyVSabdzYJZJecYE08fv3W54M5mK5xnWaASPfgsQfrnDgqQ1P67vaqSwiyx5AF/f/YYi63kOE9VOrpH3Nu0mDm97YJYqLeeXc/91hVj/Qj4j/4GeroKCg0BhRhJxGRLHRxDM/HmPzKceqg8VmZ/mhdOx2uLN/XO3jqQVG3prQng82nqV5sBc/Hkrn+bGtOZlZwsw5+3njhna8sfZUrYhTl483n+OTaZ1rhZx2kb68vlYKKJPVRrifO/4eOgqNznkS0QEeRGmKeL9XBQF+vuh3/BPV5oPitOn7iIgeei8psrIT5Y/zd1McBVD+OVnNHvmmTJ/KPS2Bqnf9KiGnrlZmyvJkJX390479xA+D7neJeBJVbc91ReoeKcR+vEt62stzZUSw1SyFXfOhUJIGqKU1Y/mdMkErqodMrYnofOUfmLqO4e34UhGEglte+TXGfAiMFxGp2SApkPs+DIunOwrrcxslV+TaT6SdJG2/OA8sVVJU2+3idBj6wpXfS+F3xWiyciBFxodr1CrendiBbWdyuXvBQca0a4Kvu45Dlwr55zVtGpwed2OXKCrNVtpH+jK2fRgdowKY+tUe7uwfx+19mqLWnLnyQRh8JFOm5egrb1ecLk4cjyAYVf27n39Wxt3H9oXxX4nA4REAem/YXZ1JU4PWTbKsglqIWNpssPyrKBQnSuebATucWA63bZRzat1TztOy8s7A4XkiuGgNkqPTbAgsuQmu+0z2telF+b2OHw5a94YFXZC2rprCqeiSnBfe4VBVDHnnYNoyET6zT4hI0+MuMBbJdWn/11BwHoIS5Frl30xavEoyGv4emsrFxbT0ZtcuoV/fFfeeIuQoKPzHVJisnMoqbfD5fckFzOzTFL3WtcE82NuNgQlu9IgLwGYDz/8wFNdWeAm3rOOojn0vbl5LpbR+Xwl145wmZ68oJgc/Atz/xyVCbD+Z4LnnUxj2EjTQpKZWq5g1qDn/+OkEsxYeYundva6Yj6SgoKDwV0ERchoReWUmJxGnLiuOpHNHv6b4eegoMpoZ1zGCDzadZVqPaM7llOGmVWO3w9IDEjjq664js7jS5b5MVhvFFSa8DFqiAtwJ9jKgVlHrACqtMPPq9e14ePERqixSwEQFuLN4vD+e84bjqXOXlfWz1QLKlssybK77TAqpulkXddn4vEx8SfxRCrM9n8kUF60bHFsshVfcQOj/RHUOThnMWCnvd2kPtJ0A302W7aYulhU042XtLGoNtBonIs/MtdLGkbrH8XzWMQkvvOErcdZkHgPfKBFx5o2FqUulqHT3l4LzcoLincNdNTppDTGXuz6eGvyi4ORP8nzBBWnJ+vmB+hOGbFYJWb3pBzAWSHGq95Tv68Fvxdmj8KeiVsl5VWQ0M75zBDvO5rH8kEyNG9UujIe+P0LzEC/yyqp4cEg8H2xyHlE/tFUIAV56vAxaXrq2DUlZJbyx9hQPDIknvaiC2+YdYPHkKHp4BtdvzQM5J5JWwrHvoe31Mv3D1fkF8vv5wDEoz5Gw7r1fyAQmkBXQU6vEmXbNhyJWJP7o/HpLpQgzt2+ChJFgMkpAMXaZtNbuRjnHvJvIaO7+j7seeW4sgMML4YbZcs4ZvCQfZ/PL4tKrIeu4CDXxw10Ls2qNBI3XbZ3S6GHne5L50+Z6ae/yCpNztiRDPnNUT1jzqOM1pVni3Ln2I3H4tBwt55MronuJAFV0yfXzdpsEnmv00jKloKDwb6PXqon0d6e4wkUQOdAixKtBEacu7rr//HbXXnAR1ZrHUNVM0wMJew9pJdcAV86bqB6g8/qP3/N/SUFREWbC8PfQ/W8PRKuH1tfCwblwarW4QRvA203HA4Ob88+fT/L2L6d5elSrP/FAFRQUFP43KJJ1I6LA2HBWjM0O2SVVfDCpE+46DU2DPBnfKZLHfzjGusQsRrdvwqJ9joJD9Rvxe/4eer6+uQuvXd8OnUbNoJYhALjrNGQUV7LySDrL7unNC9e0ZlqPaJZMbUqTVdNlSkPCqPotP3U5OEdufsqyXT9vqQLsItwAhHeAolSZ7jRxHty8SsaJLxgvYtCKe2DuWDAWwnWfSoFZ0/K0/h8SStq0v2P/Ia0k8NVugxNLZcWnrohTQ1GKTE3Y+QFEdIJON0kxOvINCXj1iRAnj+aymx2DNwx7WQSoGjrNkKlW3hHVK0suaD9JplwZvGDws9K2ojXUz/WooapEitQlM2SM8rqn4fupEDeo2hGh8GcS7GWozZca1iqUFUfSa5+z2yVD6lK+EYvNTkZRBXNmduPhofHcM6AZX07vQpcYf/ZfLMDHTceD3x/Gx03PxbwyYgI9WLxfHDzPbi4g65oF4Obn/OaBzWDYK1CSLk63jMPiShv2kpwz134kY8dBboYrisX6b7fLuXJwtvP+Ot8MkxfKOZD0s0ysGvu+iIVxA8VFM+4TEYsWzxDBpPCiCIjb35JJVxq9ZEQFNHMt4tRw8ic57vXPwA+3yrleV8SpYdcHMOBJ1xb7YS8552GpNeAdJs6/smzY+7k4Z0LbyPlc45Db+IJ8Ly9fOf/lWRnHbrdLG+jlqLUyaeu3UKnhh5lwcYcciyvRV0FBoUECPPU8NDTe5XMatYoJXaJcPvd7kFNcib0wGdXFbc4izmPJsrhjt0s7uOqyW2mPQOyj3sLm20BG4FXOpTxxQAV7XgUlQnBLycvZ96Xcq12B5iHeTOwaxRfbLrDzXN4Vt1VQUFD4K6A4choRPm4iGKhVMKhlCANaBKNCxa/n8tiYlE2gp57Ptp3nqxldMFnspBSWk1NaRU5pFSHeBrLrOHDyyqqICnAntaCi3vsYtGq83bRkFVdi0GnQa1Tc3CuWE+nF5JRWEeHnztQeMdy94CC+7jrCfNxQl5Y6VqY1ejDX328tV3quBpVGRJmKQojoIoJFaQb8+o4IM8tuFVdKXY4skG39YhwFU/45WHa7FKbd7xTx5dIumV4TNxAmzJWwwoacNadWQ/c74Ohi6DRNsjr2fCZOgms/EjfB5O+k8Cy4AGFtJRdjy2sOV0x4J2jaD1J2i9Ay/ksJY93+NuSclMK0220iVB1ZBAOfhJQ9UHxJRple/4VkbuSekuK7rrBTVSpFZU0orM0C656UdrTCZLF+e9UPe1T4/dFo1EzqFs3u8/lYqvOkagj00tOveSCDW4US4edGfKgXTy47ir+HAb1GjZteTcswH3JKqvhyxwUeHZ5ApL87o9uGsea4Y+rK2Zxybl2v4vWxqwmvOkdQRQqqJh3E8WEqEyecRi8FRsF5+T3POSk5Ut3vghGviwNn6S2QWR0SPmmh87nUZaZsv2iS47ED30gr4U3VLrm80/LfwmQRcOriHwuj3hRR98x6cdHUFDo+ESIGFV0SVw+IiFLjHIrtK66iukR2lTwdtU6cQdN/hEt7ZdS5byQ0HQCJy+DkCsdrhr0irVij3pLw8ozDci0oyZB8n6zjciw3fAVlueAZKO1m29+Sa01lsUzl0ntJRk/SSmlXNJWKIDb4Ofke5STJVL3SLLn22MxyLTBXiAjrEy7FR0WBTMDrdb/kBXkE/Lu/XgoKf1u6xgTwyLB4Ptx0Dkt1O7iXQcuHUzoR6f/75qFYLRYs5YWoKvIJ0ruhKjgvwm8NsQPBUiZCsc0qbeAz11UL0hlyHWg2GAy+aDSN8xb7UqFcm0M9rpJ5Wy3HyP3Z5pcleyy2b4Objm3fhBPpxTy8+AjrH+qPv6f+TzxQBQUFhT+XxvlX5m9KkJeevs0Dua1vHFtO5/Dx5nPY7TC0dQiLbu+Bh0HD2hNZbEzK5sd7+/BTHUfA9jN5tI30Zfd5aen5dmcyT41sxcOLj2CyOrdfPDEygU+3nmfr6Vwi/d2ZO7MbH2xI5NOpndFo1JgsVkwWG7f3i+PLbedJK6zA1rZOm1bqXsm6aGiaVKtrpCjzjao/Xhuk0PMKFXFl7+fS9qRSyT6nVRdsl4s4Nez6ULIvMo84HvOJcAT/mo3gGy1ByO0ngzFXMntGvSnF7/6vxbEQ2KzaRlECqERY2vOZjB4GceHYbVJMXtzqyPkIbi3By55BcjOXMEoK50WTIbwzXPMe/HCLtI90rZ7mo3OXAOWkn8DNRyb9RHSClB0wu7oIbnO9TPbKTQL3ANj2phTooa3/r73zDo+i6uLwb3s2vVfSSEhoAQxIKEJEo0FAwYJ8dARFEFGKih0FBRRBFFQUQYo0KaICApEqRdBAqKGmECAhBNLrbvZ+f5zsJptsgIS0Jed92IfszJ2ZM7Nz27mnAP2X0gT3+GqSTQiyTrgRS+5dkZ8C3p0BtRlmzzAz3Gwt8NX/HsCt3CKDO+L4HoGws1CgTRN7LNoXhwKtDhEtXPHTiDAUarS4lafB/gs38PIKWm20UcnR0l4LDxsppj6Qi3Rhja6eHvhsbyq0OoF2HmpodEC6fQic3LwhkSlIKXhqPVBcSBZZD5dkkUo9Q4LlpNIgODWWYi71XUAZ2v75Fig72ZApgaBIck0sz9WjpJTw60Z1Rm5RMZ23wpJWqDeNJWs1oaMMVP2X0WA86xoNyF1bUNyaPTOAFk9R3Cf99Zs+CLQbCAgJpSO/sINk1RSQUggSwMGH7uPMJlIEWbmQXHbelCnqajQpTmw8qJ49MATY+gadK3QYxfH543Xg2tFS2V1bUsysjS+Ra6OFPbD0CapPzR4Hen1O95yXTi5aWVfIuqfvt/Sc4/ZQXW7/Aj0re28g9XTJ879O7cDfX5BSihU5DHPXOFgp8eJDTdGvXRMk3sqFSi6Fl70lXG1UUNyFW5VJdDoUFORBAxlUShWUBWnQpcZCemwFVEIHNIukbHI3L5HSWaYEXo0BoAE2TwQuRtE4IHIG4NCU4vjJVSWLKlJQakvzJCFLwF6aD0uFZX2LQkilpIA/uZ4SUARFUjuvsq1YVCLBmPAATNlwAu/8ehLfDQ6FRNJAFFIMwzA1jESIyiJHMqbIysqCnZ0dMjMzYWtbsROpbRLScjF0yeEKljRutiosH9kRPb/6G05WSiwe/iA2n0jGor/JesPJSokZz4RgzM/RhlihXQKcMCY8ANtOJSM2ORtNHNR4qp0ndpy5jvXRV9C9mQu6NXNGoKs1vOwtsOxgItb8m2RYEQtwscaHfVrgoz9OY1kfO/is6VEq0KC1wOZJ5DJRFhsPYPB6ivGRnw78OpoCoeqRSGgV/exWUsZ0m0yr3YcW0P7Q4RRn5mQl2SLkKnL52PAifffvDnSdQCviCjWg05HftZ0vTfTKymfnTdY1Ue/TpEymAB75kKxpMpJKAhwWUhDTG+docvr7q8Yv13MAAEfiSURBVLQy328hWeHE7ab4OXpXEpfmgLULKaysXCi4bOzvwLmtNNGFjLJhbX+39P4Hr6fAyuXj6Ni4A09+Rff2zCIaYCYfI/evjqNJDv3EutUzAERpbJPnltBKYnEhkJkIFOaRksqjDQ2GhI6sebKTgStHKPV6kwfp95I3zBWt+q6LpkjPLcKmmKu4kVWIuLRcHIxLw+oXO2HyuuNGATsHh/kgooUbohNvITNfi3be9tDodJgXdQHLn/NEwN8TIUs6ZChf7NsNVx+eC5XKAg6nlkB55FtAYQUxchskq56vGBNJoSZLm7VDKsZXGvQLsPFlSgse0p/crG7EksVXRiIpOk+tp2DfMiW9DworoPsbVA9yU8nqxi2ErL+unyJrMs/2QMgzFKfq8kGgWU+ybLF0AJyCqb5pckvl8A8HOo2la6weQBY0/ZeSAti/e4kVj4TiW6bGUuBjvZJ00DogP4MsZm6ep2xVVi5kEZNymrJUOfgBycdJydX6WVKkZFymv6OXAgl/V/wB3dsAbZ4nBVfkLAoIbYrnV1Cg51tx5FaWdMR4f6dXSPlzbAUFaR/wM7mNFRfRhOTJBYDMPAOhmqIh1kWmkSME9WeafFqI0eRSe2RhB53SCrqz2yG/sBWFtv6QdnkV8kNfQyJXASHPUoB3a3dSIOizWhZlU78vkVI/fWhBab239aQ+t8WTwB+vQzw+HcV+PSBX1H2MmZqoi5M+nY3TRe746PEmNSzdPSIEPfPz20iB37IvxdBRVgwofzjuJubtvIDPn2uD5zvUnvsdwzBMfXJfKHK++eYbzJ49GykpKWjbti3mz5+Pjh07Vlp+3bp1+OCDD5CQkIBmzZrhs88+Q69ed8jyUkJ9DliFEFjxTyI+/O20yf0THm0GBysFHK1U2H46GWMfDkSf+fsNipun2nrisZZumPXnWVzNIEVQB18HTO/bCqnZhfjj+DUIAD1bucPHyRK/xVzD78evQacT6NPGE10CnTBlwwlczyoN7OdircLUJ1viVloKhlz7FNJLJSvrtl6UsvjsFlJcCEGDnDb/IyWGWyv6P3QYWdikxtIqf8hzpJg4sbb0xp6aDxycT1luXFsAD44Gtkw0/ZCaPEiBVVc9TwOuF7aRS9afU0pj8iitgPC3SYlzeKHx8W6tKdNM1IfAsz+SSXXCAaD9cLLiEYImiP98Ry4lbQdR9ppzWymmxu4ZFN/mz7doUvr3HONJnq0nTeo2vEhKocHrgIVdSzPxBEYAToEV5dIT/jaQsA+4FQ88u5jSHuvpNRu4FkOZrJybAZnXSJHj3orcP5yakSVUZhI9b6cAwMEfuH6GrAta9ClJ1ZxOE3elFSl4inJoEqxQk0uZVEZWQdYupmWsIxri5HHLiWsYt+oYVHIpfh4Vhoup2ZBKJZiy4aShzJAwHzhYKbHuvyuY8JAzejV3gLW8GBJtAQAJsHkiJPp03mUQAY9AEjqcfg/PUPp9Lv4FbJlkWpiWfel3O7qcvgdG0Ltt24R+4/R4qodHV5BFiXMwvbupZ2gFOvYPUgI1e5zcidYMJEWIjTtZnfiFA5b2NDlKPgHYetA5Lh8m6y+ZkrKr5d2kwJ/BvUgZk3y8VMaHJtFk6/jqUqs4hSWV0ysyvcMoJbpUTnFucm9Qu+HcnFwZfx1Nk67ITylmVt6t0vMHRgAPDAU2jAIGrqZ9zoHkRiUEtRFnN5PiSm/lN+gXUuLueJ/i2piiy3gg8DHKjLVruukyw/8Afh9PyqbQYdQmAPQ8+31HCik9Oh09h7w0ksvKme7JTFwz7rkuFmuBnBRS6ivU9I7dK8XFJecsIksq28rTFzc48m5CV5AFrU6CHLkdoLCEo5WqZq+RnUrKDZmClKByE+cv1kKXnQKtphAFQokMmSOsLeSVy6ItgtCn3ZYpqf3JuwkhkaJI5QiJtgCK4jxIpFJIZEqgKA9CpoBOroZUpwV0GkiKi6iuy5S0eFJcBKGyp3ZPIqU6oSmgsnI1yV+YDUBA6Cf0mnxAKoNEaEkWuZralaIcOqdMCUiVgK6IXCklErICLi4iazqFGrDzon6+xVPURpR1r1LZUMywo8uA+H2l2wesBP58E2LYH5A4B9bQD1U1aqJffPqD72CtVuKV8KZ3LlwfFGYDcXtpkUGmoAWJlv3ody3D93sv4XD8LWx+7SEEuJhn4GmGYZjbYR6jxNuwdu1aTJo0CQsXLkRYWBjmzZuHyMhInDt3Dq6urhXKHzx4EAMHDsTMmTPRp08frFq1Cv369cPRo0fRunXreriDuyerQItNxypPh7v1VDI+6N0SQ5eQ4qCDryPmDWiHN9Ydh6ZYoIWHDa5m5OO7IaG4mpEPmUSC2JRsTN8Si05NHRHZ2gM7zqQAEuCl5dG4fKt0Jf+Hv+Ow5WQyZjwdglHL/jNsv5FTiJwiLX74LwM9BnyGJtZzIDm1jpQk60YCPWeQ8ib1DFmp7PsCcPQjF6YHhlL68XZDaaIWsxJYP6qiBcHBr8mMdvt7pIDw7UwTnVwTwew6j6PJV7/vgKvHaIC2cXRpDBmAlBVRH5BVi62ncYrh66dIwRHci1xCrkYD/X8CDn1LK3D6SVaP92nwe34bWQ1tm0IrQy2eIkVJu8EUP6f8Sn3WNWDNYIr/o7Ci48vqUpt0MI71UZ6LUeRCkniQrIzKZiY6sogCMetdQyydgK6vk1Jt9UB6Fk0foWd05HuKGWLrRRZAXg+QlcI/39DzAUgh1PtLYOd04Fo0nUuTT3FE/MNL0sTfIZ16IyI1uwCzt58DQIGNM/KL0D3IBR/9ccZQRimT4uHmrlix/zz+GOAIZ1s1JDmJVB9ifydlgwklDgBILu0it8F1w8k1Z/TeUpckU8Tvo4C8R5eTkiT3BrDlDaAgA/B9CGg/DFjRr/T9Sz1DysrkmFI3PYCu4dyMsrTsngH0eBfYP4/e9d9eBa6WtgdQ2QAjttIEp2y68pSTpJx9bgnw65hSJc1/i8kC78wmwCmI3ukNo4zvI+kwKWue/BroNomCep/bRi5Tix+nut3rC3rHC7OMj734Fyleg3tRLBzXFjQhi/6pNI5NmwHUFmwcTVZpchUACSmOK0NpTQqsYysqLxOzijJ4+YSVWggCZG1UkFGqyNHkU33+9eXSbGRqB1JgB/Sga93PZKeUtD3f0jO19wEe/YjuvbouaDmp9N4fWkBKS1svck1t9njDdmvTFADXT0H8+RakV6OhlMphFdwPNzq+iSsKdwS52cBCcY+WXAWZ1Ldtf5esOhWW1L92GU/9oZ7s69D9twTSw99BWZAJpa0XtJ3exgFpKFoH+sHfudx7mZEEcWItJP98S/XbypliQnm1hyR+L1Q2HpD8PYfGBiobWgTx6QTJjg8ge24JuT0e+Z4m6dauZNVm6wW4h0ByfBUphex9qH8/tIDqikJN44uAR4C/PoTkyfm0L3QYJVw4vYnqtNIaCBtDCxoSKb0TSmtg84QSy9S2FEh942gq0/Z/5ALt3YmsFcsqcQCScdNYCghfVpGTcx3iqQXQWbnAnO3tErUOeMSi8pTv9Y7KhhaemoaTQufYCuDSHqDH22RRWsLwLn44n5qDV34+ik3jukKtNOdfhWEYpiINICT9vTF37ly89NJLeOGFF9CyZUssXLgQlpaWWLJkicnyX331FXr27Ik333wTLVq0wPTp0xEaGooFCxbUseRVRyaVQKWo/CdTK2T4N/GWoayvkxU2HbuGhUPa47shoXCztcCsP88i6VY+xv58FKNXROPLqPM4ejkdjzR3w4JdF3Dheg7OJGcbKXH0XM3IR3RiOsL8jQfCiWl5sLdU4PHFF3Gz8zsQQzdRppyR22gldMnjNGCK/R24fIBWya/8CyTuJyWJpT1N5E6uq6jEAYC0C+T2BFAgwYQD5MrkHVZaxtaLJj4XdtBq/u4ZpFiJ/d1YiVOWfxfRYLI82iKKSROzirJPbXkDuLSzdMKbm0b3Y+9LFgr5GbT9zymk2GjSkSYhZzebvm52Mg3Y935W8X41BbefuKlsaKAJ0LFlM2alJ9CKvn6SnHeTLIsuHyLFkhB0HxtfpCxEQInCbTilj973eakSB6DJ5ZqBpIzLTye3LQtb+g3i9lCmsJuXKpe1kaHR6pBws/T3dLJS4XDcTajKxHB40N8BR+JvYWa4FVzUEkhOb6RsHGc2kULuToHA9Wlu824Bcftowl8ZFva04t6yL8VK2vcFKRAAoMML9L6WVSJa2NMEqqwSR0/aBeDSbnpv1o+kcx782liJA5D82SnGShw9hdnAga9pkqWnIJOuaeNB7/LeWabvJTuFFKSuLem7XzfgTEndtvUia7vyShw9x1cDrfoBLkH0rP/5tvQ5awtpwn/md5roATSxlSmpDTCFREKBn/X3VBkFmWTxtH5UaSB1e19SLJXNvpKeQFm+yqaUz08n66L7vX7lpVOWwT0z6XkB1KZuGFny+1YSC+125GcAUVPJUkr/3LOukqLsxFpjV96Gxs0LwJJISPTvh04LVex6NNn0HLTpV3D+eg1MrhP2UyBz/bulyaM6sX4kKcAAID8dYvs7kO6dVfq7ZF2F447x6F64D19HnUVyRpm2KucGxIGvINk1vbT/yU2jrHDntwEeoZBsKeNqXZhNSpvjq4DeX5Dy5e8vSutTTir1NzfOkdLYJ4zaqqvRdE59XdHkk1L2vyUU3+rXl8kq95/vyP1an72yKIfOn3ICOLed+tCNo0rbjIcmApteoXIdX6L3Rwig7UDgv59MP0e99U6ZrHbCtSXg2hIyM45Hl5WdjVvCBm4NJDzObdErdDq9SlakmycZxUe0UMgw4dFmSLiZi7fWH8d94IDAMAxjhFkrcoqKihAdHY2IiAjDNqlUioiICBw6dMjkMYcOHTIqDwCRkZGVlm9IWKvkeKGLX6X7B3fyxbZTlOGmo78j9l9Mw+5zqRi17D/cyinCD/soXk5mfhG87EszPcilUqTnFeH4lUyE+Ttiz9nUSq+x59wNhDU1VuT4OFkiM0+DAQ96I1vuiDVJjkiybIni2C00aBy4mlwxWjxFrhuWzrSafuIXWtFX2VC8iUpv3LV0MGnXhBQTm8bQyvboPaQ0evht4N/FpHwBaNJ3NZoGgpVx8yKZT5dFKqMVeZmCJnSiuGJWHj0H5lFsGisnmtjptBSYuSADgKTUUsYUGYk0KfV/2Hj72T/ITLgy2vyP/MElUprE6yf2AFkb6DOHleXoMnJr05N3iyyF9IqwtPOAJsf09W7F0Tn1g9WDCyiVOkCD6Ys7K5e1kSGXSeFua2H4rtXpMH1rLAY8WOqfr1bI0NpZBpf4X2kC79mOfnM9dwouWdb9Ydc0cvmrjLYDaUW69bMVFStyVcUsbf7daNJVGTkpwIUomuz4dqbYL+VxbQFc/bfycyT8TW5helyCaULUcxagtieFUWWknAIgobrZok9pIGFLJ2OruvIU5ZJLoUwFnPzFdJnY36g9CYoELmwHjvwIdH+LlC7leeRDUgio7SnzXWW07Atc+gsozKTn3fZ/QJ+5FHTZomSipy0CDn9vOni7EOSaWVa5er+Rm1oax6s8Oz8CcpKrcc4bpCAwxe4ZpBRsiBRkATunlSofypKZBO+cE/jpQDwy8ytZmLgbslPIos0Ulw+VJh/IuQGJqfoNwO7QLAxppcT51DJKpfxbkESbXjzDke8rjwd18S+yAiqfqU7P4YVA815A8kmKg/XPt6bL6fup7GRSnpqKfwXQ8Q++ABz6plSJbWFHCiG9kluqKP1bbV/qkm2KrKvU/wPU9tl6QmJOLnwmOHMpAQDgY2dGBvu2HkCnMbTg99dHRok2vB0tMTY8AH+cSDZYzDIMw9wvmLUiJy0tDcXFxXBzM06v7ObmhpQU04O1lJSUKpUvLCxEVlaW0ac+ecDHHhEtKrqMdfR3xAPe9jh/nSbkdmoFbmSXTvIVcimuptMK2uojSXi5jO+zj6Ml/okjSx6tTkB5mywQSrkU2jJplV2sVWjv64DvhrTHI81dcepKJj7ekYgxv6cgU6ei1bJ1I2jAI1PQ6vfiCFLODPud4kwkHgB8Opv20QdI6ROzipQszfvQgCk3jbLyXDtGK7q/jzfOVAVQ4GG99YopnEpiZZSl7UCazBZmU9rw62dMHwuQMsY9BNj6JqUo1mdQ2P4erbwrrSo/1vMBigOhtKL4I3puXqLn0Oyxise0eIriA+TfKpFzu/H+TmMpW1V5ijUVrZKSjlA8ID2FORQQ0hTXT5XKmJ1Mg1s98XtpMloHNLS6WB5XGxXGP1oaFyE9T4PMPA2upuejf3sKGnn6WhZC3WVQWNiQSb6mwNgq5up/lSsHgiLJGk1PznVA7USufeXx7Qq06E2WNyrbO1v6ADSBqcx6DSClht7ty9Q7VR26TqD6sup5msCWjRtTHrsm1IY4+JFVmGMAbc9MKs1KZwobd3LJyLpaebY7IUjx+vinpOBsFkHKnEenkhtkSH9yaxu0ltK6K61JUfzAENP13CmA2jiVLfDcT5TZSldM7l8FWaWKa00uWQlURuoZspZrYNRYXbydoj0/vVSBXxUqU7wDZIGhn6Q3NAqzaeGjEpySdkCrE8grrIaVUtlrZCRWvv9KiYXdzYuVlynIhIM0H6eulvnNc1Irr1vFGrKWMIXKluK9VUZRyQLDpZ2Ayrry98E5kCxI1fbUR1WGJp/kLHt/SmvqU02Rnkjjjcpwaw3cjIPwD4cYsBISB9/Ky9YSNd0vnoq7ChWK4OVkXzMC1hVyC3LXd/QHdn1C1m0lhDV1wuAwH3y75xLm77zNYgHDMIyZYdaKnLpg5syZsLOzM3y8ves3+r2LjQVmPtMGq14KQ582HniitTtmP9cG/dp5olBTDGsVraJcuJ6DNk1KzXsT0nIR7E6BAE9ezcSV9Hx82KclXGxUyC7UwMmKgsTtOpuKXiGVryj1aeOBv2JphSrAxQpfD2yHTceScCOnEG+tP4G03CI4W6sQm5yFbK/udJC2kFyeTm0oHbgnn6BtXqE0eTq7GXj6e3LvKEu7wZROuU1/YOQOsh5xDym1XDheYu1jisuHaAJWLgCegS4TSuPRqGxoUvnAMFIaxawEgnrePkCmyobkSToMnNwADNtEcWScg2hQ2Xm86eOahNFxO94j0+xuk+lj5ULKqjO/UQye/62miWLoMAqc7BVKE39IyHXrv8V0Pgs7oM+X9EzLZy/SU9YFC6AJc2HJoFgipbgRopKBuGMAcL0kWK/SynjF2Ma94rlriYZWF8sjkUjQs5U7XujqB6kEOHjpJnoEu+LdX0/i+Qe9seqlMHT0d4RMrqDf0dazovLy4AJy8QmMKHtiSmMfOpxcg/SobGhC0m4w8NIuCvIdNoaCYLfsCyx6FNg727T7lbagYqyQpMNAwKOV36BtE3KB0stkKiBtaiwpWSrDrxul/Lb3oQxsSUdKJ1WHvwe6vGb6OLmK4vdoCyl+ypV/SbElU9KEX6cl5Ykpwt8GFNa3d0MDSPGy7R1Kva5QAydWA7s/IVfJRz4AfLqQVVCLJ6n9+esjsqB4fkWJklVJE9MHX6K4U7be1OatG0Fxf06uAyDIgtC25N1VWFLGr8pwCgKUDc/Hocbq4p1+k8qU+7ejfB9S4ZwWt99fX0hlZF1WCYWW7pCA0itXG5mSAglXhnXJIlFZZb0JdDIlAl3LuAAr1JUXBirvIzT5d34HZAoK/A0JtTumyEunxZOiXArEXxkSCfV3LmViu+WmkrWPHp2m9B06upRiB5nC2g3wewii/1LgqfmQVNb+1DI13S+euJYNX8l1yNQNI4FAlZDJgbaDaSz118dG7qp92niif/smmBN1Hu9vOoki7W0sphmGYcwEs1bkODs7QyaT4fp1Y9PX69evw93ddNYLd3f3KpV/5513kJmZafgkJSXVjPD3gIuNCl0CnPHp063xUKAzvthxDu/+ego/7o/HrGdCIJEAl27koImDJVxtaCC8PvoKRnT1M5zjh31x2HwiGW9FBuPdJ1qge5AzVHIp4tNyYaGQoXNAxQFlR39HdA9ywbS+rbFoWHu8/mgzuNtZIKK5O4JcrTHvf+3Q0sMGL3T1g04AK04VIvOhDyregLUbuUJ5tiuJfSMh1x2ZijI6DVwLDFgBvLiTJo5bJpFJeE4qBTU8tQF45geKjZF0hCaUIc8ZX0NpTQFMT6yjia11GSsmhSUQOZMCuD67mCZWfeaRcuXQAuB/q2gy5hZClkKVDVLbDSp1L8krCTx84xxZByT9A9i40cRUv2IvkQIt+gKPfVR6jqj3KUaBjScQ8THQfzkFf932DvDbKzQQyU4Bfn+N3GSatAcu7qA4BEN+BUZupwmxlQspxkzhFEgxUsrSsl+pRU/r50pSPZtA7UCDXn1Ax7YDjV0hHhha+eC6hmmIdbE8TtYqvPJwADa+0gXtfewx6fEg2FsqMfbnaEAAYf4OSIcdtA7+pIi48h8pKvVo8ihWhXsIMPQ3Chz8wjbKOLZ+pPHKdpfXSKFxagOgtCWFTkh/Ui78+RaVTY8nxaB/uLGgh78HHptm/Lvr41c0ebDijakd6N1r/Sx9P/ELBQctj05LdaDTKxX3qWyAxz6md6/Hu+QKGV0m/sTx1RSMts3AcsfZAs8uof93fkzPSJ+a/JlF1J5ETQWemE3tg/59tLAnixqXYGD9cFqhrUzJ5NqSVvsvRtFzzU2j+FgDf6HrFWaRctejLbl5nFhDx2kLyMImuBfJMmgtKXOVNoBLM/rtnvmRLKMe/wQYe4hcuOQlE1u5Cuj8SuX1r/vk21v21RM1Vhcd/SpXvPh3JzfcqmLrVbllV5OOt1WW1CvWbvQuVEJa4HNo6WELZ5t7yF5l5Qy0etb0Prmq1O3R3qfSoNDFPl1xIFmC1p5lJvqWzsbKkLI4B5UsQJhAaUULEdZupvd7h1EmxrYDyFox4BHT5TISgYCHydqmKIes90wR+DhlmOw0trTOFWvI8lZ/7iOLqJ2SSMhCNvk4BYu3KpOl0aczxPA/ICxdoHMKqhdLHD013S+euCmBvzKDUq+bI3IlEDqU/t45HdCWWqM+E9oEL3VrijVHktD3m/04djm9kpMwDMOYB2affjwsLAwdO3bE/PnzAQA6nQ4+Pj549dVX8fbbb1coP2DAAOTl5eGPP0rjUnTp0gVt2rTBwoWVpHwuQ0NLeXwztxDxN3Lx8z+JyC0qxqsPB0Aul2LFoURkF2jxYjd/zPvrAvZduIH+7b3R1tsOc3ecx81ccoto5mqND/u0xK6z1xHq64jJvxyHRAJ82KclFHIpos5chxACj7V0g4VCBmdrpSFrhqOVEiq5FEVagWKdDgqZFJpiHeQyKWb9eRZ/nkrB611d8XyAFi5nV0CZn4rioF4o9n8EQqaA9O8voDj+M62QDl4HIZEAqWchUdnQhEmng9AVQVKUR4MKhSUEStOPSqCjAWJhNlk35N+i7DjWrjR4PLeNlB7NepKfvT4miNqBTNiPLqeMNf7dKebFzQuUpaJZSSacuH2Ad0dAaCtmxAmKBDq+TLFHQoeRDCd+AR6aQC5fcXuoXLdJpCjR5JNpuK6YrC7aDSbZDn1DljBNHwHaDSTlkcoGKC6k+DPHV9GA84GhlCp87WCaLLcdSNYN2cn0ObeVskhtGmOczcvKmQJDb55YGv+g22SybIj+ic7bcTSt1G5/1zhWgY0HKbViVlFg6IBH6V43jKT9T8wmS4R6SkPe0OpiWbIKNEjLLsSJKxkIdLXB/gs3EJ2YjqdDm8DLXg1vZTbsU49AKpVS9rL9c4DL/xiOF54PQPLI+5Tt6LHppDDYP5diwTgFAJ3GkSuQgy+w7gWaeA3aAOTfBHKvl2QuK3EXsLADBq6hAN1lXVnaDQE6vkjvYNp5UhY9OIrcjBL2UzwZTR4Q3Jviu5z4hZQwV/+jgKNDNtJEZ99npGSVyskSKHwKKf4snYFjy+l99A+n4MG5qRQ8tNMYuq4+iLiDPwXVdvCnVNwSCZB2jpQ3Nu5k4ZZwAOjyKq2623gAf75JCs3O4wEHH0AiJyVt/k26d6m8JO7OeYpdpbYnpcq2KcaxeBz8yeIt8QC1BTfjqKyVM01uc29SvcpPh67XXMC3M6SFWaXKJH1MLJU1tUdWLoBdmew/d6IojzL6/fZKqeuI0pri6QT3ovaggVPtuqgrJmXmin7Ggd/tfcj11tG/6sLodORmu+zJUtccgBQ8w/+o3HKrIZCdAvHbq5BcjDLanNFjJjajOx5/IBCuZeJwVYvMJGDl8+S2p0empDbCrxtNhHU6spxb3tf4Gdr7IOnJNci19Eawuw0keqWpTgeRchKSn58uDXYMUF0YvJ5+29/GUSwbPUorssI9vpYm3htHG7u96a32bsWTG6lvVwCCXJnLZpSzsAee/o4WJrzDKDhyzxkUf6+sm5VHW4rFlZ0CZF4jJeIfr5G8chW5QCb8TeOCoJ7Uxx5fQ+1QcG+IVv2oriosAJUtJPYNyyJUz730i5l5GrSdtgNj7A8jvFuPOx/QkMlKpgULrwdo4aCM63h8Wi5+2HcJCTfzEB7kgoEdvREe5MpZrRiGMTvMXpGzdu1aDB8+HN9//z06duyIefPm4ZdffsHZs2fh5uaGYcOGwcvLCzNnUpaegwcPIjw8HLNmzULv3r2xZs0azJgx467TjzfUyWOxTgedABQyWkXRFBejQKODhVyKtJwi5BZpUawDnK2VKNAWIz1XA7lMAmdrFZRyKfIKtRAQyCvSIToxHdczC9AlwBle9hYo0BajQFsMa5UCCqkEN7ILoVbKoZRLkVOoRXaBBk5WKjhZK2FvSW5Mt3KLcD2rAIfjb8FOLcdDTR1gqwQkCgsoZFJIJBLoCnOBvJsQumIUStVIhw2sZICVyILQFCBPagmZTAGlJhvFkCBdWEEiVUKplENSrIWD7iYkEKQI0WoACQAhIGQKSKRKMlUvLqSJsiYHyMsAUAzYeFHZwmwawOrdTIqLKeuT3u2lKIeUOxaOkNh5kL983k0Il+Y0wSzWlJiMSwFdISl/FFaA0EFSXESTUbkloNNAZF+jia1EQoEhi7XkXqErpEGqwpJkkFvSBF0iIeVVYR7JqrIlRZW2AJCrSfmVe4OUTJYutF1bSNZDaWdphdG5GeDehiZL5/6kAWjTHnTuvJs0WVQ7AroiKiNTAUXZ5J6ldqRBuCaf3KrcWtNg9+JOktW3M92PVf2tbjfUumgKnU4gt0gLuVQCmVQKQECbkwaVNhtSoaPfQyqld9LKFVqVA3RaDWRFmZCqLCGEBFIJ6F3XFNBvqckHEg8BrsHkeiBTlLqNaAuBrGuQ5N2AsPclix2plK5z4yxNkux9yOUIOlJSypSkDNGUBAeWSkvqF4DiIgiZHIC0pM7kA1eP0aRYbV86wVFa0SRQFFPZglulihegdLIuV5O82kJyZZBb0DmLNXQ+TSGd09KJFLfaIiovU5F82kIKoJqbRkonCztaURc6SIo1EEJXGrAcEjqmIJPeeaEjpW7mVcDBm951hZoG+rlpkEiltE2movajIIOOsXQqec5lXFOKi4G8G6VWQpYugKoaFjTFWgrsm5NKii1rN7Loq8wttIFxT3WxWEvWYEmHSYnm3YEspGyroAwrj66Yft+r/wI3LtBkzq11xeD2DZHcG9ClX4a4tAsauTUKfB5GltwRljb2cLa+B2ucsmSnkEtj4kF6Jj5dyY24rCtbyTPUXfkX4uZFaN1DoXEKRp7KBS42FqVKnDLlRXo8WbCkxtLzdmtFVrPaAlKSZiQB16LJDcq1OXD1OKC2o3KafFIM37oIiXMwtRlyC8qgZeMKpF+mPlKmJAVN2nnAwQ8Sp2bU3+WlQfh3o3qYcZkUPIXZQPY1skq1dKZ6mp9B/ZZUAUCULPBISvpfC+r3dcXUdsgtgOJCCKUtpFZ3cAFrINxLXfwt5ipeXxODBX4H4BRiIk6fuXHjLHB0BfUNXV8zUubodAIHLqVh2+kUxN3IhUImQZsm9mjbxB4tPGwQ7G6DZq42rNxhGKZBY/aKHABYsGABZs+ejZSUFLRr1w5ff/01wsIoI8/DDz8MPz8/LF261FB+3bp1eP/995GQkIBmzZrh888/R69eve7qWuY0eWSY+xmuiwzTMOC6yDANg3upi68uP4QzsafwaWgu4Nm2liSsY67FUHyyJh0oDqI+Y2AZkjPycfxKBs5dz0bizTwkZ5IroASU9aq5uw19PGzRytMWPo6WFRWZDMMw9cB9ocipS3jAyjANA66LDNMw4LrIMA2D6tbFAk0xOkz7Ez2L9+LZR7rcMeC1WXHjLHByPf3t351iM2oLyDItO4WsRpVWFOfJKxQFzq1xNUuDpFt59EnPR9KtPGTkawAAthZytPd1QKemTugS4IxWnraQSlmxwzBM3XOb9AUMwzAMwzAMw9zPrDp8GXka4CHrRMDiifoWp2ZxaU7WOAl/A1ejgYt/kYue2qHEjdyJXPvi9wJnNsFCZYMAv4cQ4NMZCGhJrsAAMvKKkHAzF5du5OJ8SjbmRp3HzD/Pwt5SgS4BTujU1AmhPg4IcrOBUm6mwaIZhjErWJHDMAzDMAzDMI2QW7lF+HbPRTwkPwM3T786y0RZp6isgeAn6FMZQpCFTnIMcPkwxRaUSCngvtoR9jIl2kllaCdXAk6O0Po1wXlpAE5lW+J0cjZ2nL4OrU5ALpXA29ESfk6W8LRXw8POAu52anjaWcDLQQ1Pe7UhniXDMMy9wIochmEYhmEYhmlkXMvIxysrj0JTkIf+kl1AkxfrW6T6QyKhoN+2HpS5LCeVUtvn3igJgl0EaEuC5addhDxvK1rqitFSaQW4t0aRbwskSH2QWGiDa/lS3MgtQHxaDm7lapBVoDVcRioB3Gwt4GWvhpu1HHZKHdRSHaQQKJYqoJWqoBESCAFIJBKo5FKolTJYq+SwUyvgaKWEg6USTtZKOFlRkhEZu3YxTKOEFTlVRB9SKCsr6w4lGYapDjY2NncVSJDrIsPULlwXGaZhUBt1cfLGWESdTQMAdJBcxH5pa+hOpAJIvSdZ7y9sSj7lkAGw1kFSmAVJfjoQrwPiLwC4AACwK/kElhTXSOW4KWyRBjvcEPZIznQwBFVmTCMBIJdJIJNIAPoHgBJpCgHohIC2WKCqgV4lAGRSCWRSCSQg/Z2XvQVmPBWM5m7Wdzz+busiw9QFHOy4ily5cgXe3t71LQbD3LfcbZBGrosMU7twXWSYhkFt1EW3QbNg4d0aolgLG0ke5FKyAmGqT20bxuggQQ7UKAanRa9p0nf/hKwjG+5YjoP6Mw0JVuRUEZ1Oh2vXrkEIAR8fHyQlJTX4Cp2VlQVvb2+zkBVgeWubhi7v3a526Oti+fIN/f5qm8Z8/3zvNXvv91oX64vG/B7UJPwca4aaeI71XRf5Xaga/Lyqhjk9r4bSzzEMwK5VVUYqlaJJkyYGs1VbW9sG3+joMSdZAZa3tjE3ecujr4uVYe73d6805vvne6/be79TXawvGvN7UJPwc6wZ6uI51nZd5HehavDzqhr8vBimanDYdIZhGIZhGIZhGIZhGDOBFTkMwzAMwzAMwzAMwzBmAityqolKpcLUqVOhUqnqW5Q7Yk6yAixvbWNu8laV+/3+7kRjvn++98Z57+XhZ1Ez8HOsGe6H53g/3ENdws+ravDzYpjqwcGOGYZhGIZhGIZhGIZhzAS2yGEYhmEYhmEYhmEYhjETWJHDMAzDMAzDMAzDMAxjJrAih2EYhmEYhmEYhmEYxkxgRQ7DMAzDMAzDMAzDMIyZwIqc2/DNN9/Az88PFhYWCAsLw5EjR25bft26dWjevDksLCwQEhKCrVu31pGkVZN10aJF6NatGxwcHODg4ICIiIg73ltNU9Vnq2fNmjWQSCTo169f7QpYjqrKm5GRgXHjxsHDwwMqlQpBQUEN9n0AgHnz5iE4OBhqtRre3t6YOHEiCgoK6kjaqmNOdbM2MLf6XpOYW9tRk5hbO1QXfPrpp+jSpQssLS1hb29vsszly5fRu3dvWFpawtXVFW+++Sa0Wq1RmT179iA0NBQqlQqBgYFYunRp7QvfgPHz84NEIjH6zJo1y6jMiRMn0K1bN1hYWMDb2xuff/55PUnbsKlum1XXNPZ+tao05n64OjTmvpthag3BmGTNmjVCqVSKJUuWiNOnT4uXXnpJ2Nvbi+vXr5ssf+DAASGTycTnn38uzpw5I95//32hUCjEyZMnG5ysgwYNEt988404duyYiI2NFSNGjBB2dnbiypUrtS5rdeTVEx8fL7y8vES3bt1E375960RWIaoub2FhoejQoYPo1auX2L9/v4iPjxd79uwRMTExDVLelStXCpVKJVauXCni4+PF9u3bhYeHh5g4cWKdyFtVzKlu1gbmVt9rEnNrO2oSc2uH6ooPP/xQzJ07V0yaNEnY2dlV2K/VakXr1q1FRESEOHbsmNi6datwdnYW77zzjqFMXFycsLS0FJMmTRJnzpwR8+fPFzKZTGzbtq0O76Rh4evrK6ZNmyaSk5MNn5ycHMP+zMxM4ebmJgYPHixOnTolVq9eLdRqtfj+++/rUeqGR3XbrLqmsferVaUx98PVoTH33QxTm7AipxI6duwoxo0bZ/heXFwsPD09xcyZM02Wf/7550Xv3r2NtoWFhYmXX365VuUUouqylker1QobGxuxbNmy2hLRiOrIq9VqRZcuXcSPP/4ohg8fXqcNelXl/e6770TTpk1FUVFRXYloRFXlHTdunHjkkUeMtk2aNEl07dq1VuWsLuZUN2sDc6vvNYm5tR01ibm1Q3XNTz/9ZFKRs3XrViGVSkVKSoph23fffSdsbW1FYWGhEEKIt956S7Rq1crouAEDBojIyMhalbkh4+vrK7788stK93/77bfCwcHB8AyFEGLKlCkiODi4DqQzH+61va4rGnu/WlUacz9cHRpz380wtQm7VpmgqKgI0dHRiIiIMGyTSqWIiIjAoUOHTB5z6NAho/IAEBkZWWn5+pS1PHl5edBoNHB0dKwtMQ1UV95p06bB1dUVo0aNqnUZy1IdeX///Xd07twZ48aNg5ubG1q3bo0ZM2aguLi4QcrbpUsXREdHG8xc4+LisHXrVvTq1avW5a0q5lQ3awNzq+81ibm1HTWJubVDDYlDhw4hJCQEbm5uhm2RkZHIysrC6dOnDWXulzaiJpk1axacnJzwwAMPYPbs2UbuaIcOHUL37t2hVCoN2yIjI3Hu3Dmkp6fXh7gNjppor+uCxt6vVpXG3A9Xh8bcdzNMbSOvbwEaImlpaSguLjYa+AGAm5sbzp49a/KYlJQUk+VTUlJqTU6gerKWZ8qUKfD09KzQKdcG1ZF3//79WLx4MWJiYmpdvvJUR964uDjs2rULgwcPxtatW3Hx4kW88sor0Gg0mDp1aoOTd9CgQUhLS8NDDz0EIQS0Wi3GjBmDd999t1ZlrQ7mVDdrA3Or7zWJubUdNYm5tUMNicrqv37f7cpkZWUhPz8farW6boRtQLz22msIDQ2Fo6MjDh48iHfeeQfJycmYO3cuAHpm/v7+RseUfa4ODg51LnNDoyba67qgsferVaUx98PVoTH33QxT27BFTiNn1qxZWLNmDX799VdYWFjUtzgVyM7OxtChQ7Fo0SI4OzvXtzh3hU6ng6urK3744Qe0b98eAwYMwHvvvYeFCxfWt2gm2bNnD2bMmIFvv/0WR48excaNG7FlyxZMnz69vkVjapiGXt9rEnNsO2oSc2uHyvL2229XCLRb/tOQJsLmQlWe66RJk/Dwww+jTZs2GDNmDObMmYP58+ejsLCwnu+CYcybxtQPV4fG3nczTFVgixwTODs7QyaT4fr160bbr1+/Dnd3d5PHuLu7V6l8TVEdWfV88cUXmDVrFv766y+0adOmNsU0UFV5L126hISEBDz55JOGbTqdDgAgl8tx7tw5BAQENBh5AcDDwwMKhQIymcywrUWLFkhJSUFRUZGRKXpDkPeDDz7A0KFD8eKLLwIAQkJCkJubi9GjR+O9996DVNpw9L3mVDdrA3Or7zWJubUdNYm5tUP3yuTJkzFixIjblmnatOldncvd3b1CdhT9c9Q/u8raCFtb2/vKGudenmtYWBi0Wi0SEhIQHBxc6TMDYJZta21wL+11XdLY+9Wq0pj74erQmPtuhqltGs4MrQGhVCrRvn177Ny507BNp9Nh586d6Ny5s8ljOnfubFQeAKKioiotX5+yAsDnn3+O6dOnY9u2bejQoUOtyliWqsrbvHlznDx5EjExMYbPU089hR49eiAmJgbe3t4NSl4A6Nq1Ky5evGjoeADg/Pnz8PDwqPXJU3XkzcvLq6Cs0U/+hBC1J2w1MKe6WRuYW32vScyt7ahJzK0duldcXFzQvHnz237u9h46d+6MkydPIjU11bAtKioKtra2aNmypaHM/dJG3I57ea4xMTGQSqVwdXUFQM9s37590Gg0hjJRUVEIDg5mt6oSqtte1zWNvV+tKo25H64OjbnvZphap76jLTdU1qxZI1QqlVi6dKk4c+aMGD16tLC3tzdkvhg6dKh4++23DeUPHDgg5HK5+OKLL0RsbKyYOnVqnaYfr4qss2bNEkqlUqxfv94otWh2dnaty1odectT19Hrqyrv5cuXhY2NjXj11VfFuXPnxObNm4Wrq6v45JNPGqS8U6dOFTY2NmL16tUiLi5O7NixQwQEBIjnn3++TuStKuZUN2sDc6vvNYm5tR01ibm1Q3VFYmKiOHbsmPj444+FtbW1OHbsmDh27Jjh/danH3/88cdFTEyM2LZtm3BxcTGZfvzNN98UsbGx4ptvvmnU6ccPHjwovvzySxETEyMuXbokfv75Z+Hi4iKGDRtmKJORkSHc3NzE0KFDxalTp8SaNWuEpaUlpx8vx53qbUOhsferVaUx98PVoTH33QxTm7Ai5zbMnz9f+Pj4CKVSKTp27Cj++ecfw77w8HAxfPhwo/K//PKLCAoKEkqlUrRq1Ups2bKlQcrq6+srAFT4TJ06tUHKW576aNCrKu/BgwdFWFiYUKlUomnTpuLTTz8VWq22Qcqr0WjERx99JAICAoSFhYXw9vYWr7zyikhPT68zeauKOdXN2sDc6ntNYm5tR01ibu1QXTB8+HCT7/fu3bsNZRISEsQTTzwh1Gq1cHZ2FpMnTxYajcboPLt37xbt2rUTSqVSNG3aVPz00091eyMNiOjoaBEWFibs7OyEhYWFaNGihZgxY4YoKCgwKnf8+HHx0EMPCZVKJby8vMSsWbPqSeKGze3qbUOisferVaUx98PVoTH33QxTW0iEaGC+EwzDMAzDMAzDMAzDMIxJOEYOwzAMwzAMwzAMwzCMmcCKHIZhGIZhGIZhGIZhGDOBFTkMwzAMwzAMwzAMwzBmAityGIZhGIZhGIZhGIZhzARW5DAMwzAMwzAMwzAMw5gJrMhhGIZhGIZhGIZhGIYxE1iRwzAMwzAMwzAMwzAMYyawIoepcZYuXQp7e/sqHTNixAj069evVuRpSPj5+WHevHn1LQbD3Lc8/PDDmDBhQn2LwTBmy549eyCRSJCRkVHtc3A9ZJjq8dFHH6Fdu3b1LQbDMGYAK3KYu6YyZUv5Qd+AAQNw/vz5uhXuDtztwFRfTv9xcXFBr169cPLkySpdrzJl1r///ovRo0dX6VxM4+bQoUOQyWTo3bt3fYtSZ2zevBnh4eGwsbGBpaUlHnzwQSxdutSoTE1MNhmmrinbv5j6fPTRRxWOycvLwzvvvIOAgABYWFjAxcUF4eHh+O233wxlqrtIYErh0qVLFyQnJ8POzu6Ox1dWDzdu3Ijp06dXWR6GaWiMGDHCUD+VSiUCAwMxbdo0aLVao3Jz5syBg4MDCgoKKpwjLy8Ptra2+Prrr+tKbIZhGgGsyGFqHLVaDVdX1/oW4544d+4ckpOTsX37dhQWFqJ3794oKiq65/O6uLjA0tKyBiRkGguLFy/G+PHjsW/fPly7dq1WryWEqDA4rWvmz5+Pvn37omvXrjh8+DBOnDiB//3vfxgzZgzeeOONepGpJuo+wwBAcnKy4TNv3jzY2toabTP1jo8ZMwYbN27E/PnzcfbsWWzbtg3PPfccbt68WSsyKpVKuLu7QyKRVPscjo6OsLGxqUGpGKb+6NmzJ5KTk3HhwgVMnjwZH330EWbPnm1UZujQocjNzcXGjRsrHL9+/XoUFRVhyJAhdSUywzCNAFbkMDWOKWuUTz75BK6urrCxscGLL76It99+26Tp6BdffAEPDw84OTlh3Lhx0Gg0hn2FhYV444034OXlBSsrK4SFhWHPnj2G/YmJiXjyySfh4OAAKysrtGrVClu3bkVCQgJ69OgBAHBwcIBEIsGIESNuew+urq5wd3dHaGgoJkyYgKSkJJw9e9awf+7cuQgJCYGVlRW8vb3xyiuvICcnBwCtUL7wwgvIzMyssMpaftX08uXL6Nu3L6ytrWFra4vnn38e169fv/NDZhoFOTk5WLt2LcaOHYvevXsbWaUMGjQIAwYMMCqv0Wjg7OyM5cuXAwB0Oh1mzpwJf39/qNVqtG3bFuvXrzeU16+m//nnn2jfvj1UKhX279+PS5cuoW/fvnBzc4O1tTUefPBB/PXXX0bXSk5ORu/evaFWq+Hv749Vq1ZVeL8zMjLw4osvwsXFBba2tnjkkUdw/PjxSu83KSkJkydPxoQJEzBjxgy0bNkSgYGBmDx5MmbPno05c+bg8OHDd6zTOp0Ob731FhwdHeHu7l7ByuFOculN23/88Uf4+/vDwsICAA3GQ0JCoFar4eTkhIiICOTm5lb+AzJMOdzd3Q0fOzs7SCQSo23W1tYVjvn999/x7rvvolevXvDz80P79u0xfvx4jBw5EgBZ1SQmJmLixImGPgcAbt68iYEDB8LLywuWlpYICQnB6tWrDecdMWIE9u7di6+++spwXEJCQgUrm+r0reUtfQoLCzFlyhR4e3tDpVIhMDAQixcvroUnzDA1j0qlgru7O3x9fTF27FhERETg999/Nyrj6uqKJ598EkuWLKlw/JIlS9CvXz84OjpiypQpCAoKgqWlJZo2bYoPPvjAaKxbHlNWc/369TPq8+40PmYY5v6EFTlMrbNy5Up8+umn+OyzzxAdHQ0fHx989913Fcrt3r0bly5dwu7du7Fs2TIsXbrUaOL66quv4tChQ1izZg1OnDiB/v37o2fPnrhw4QIAYNy4cSgsLMS+fftw8uRJfPbZZ7C2toa3tzc2bNgAoNTS5quvvror2TMzM7FmzRoAtEqpRyqV4uuvv8bp06exbNky7Nq1C2+99RYAMksvv9JqapVVp9Ohb9++uHXrFvbu3YuoqCjExcVVmJwzjZdffvkFzZs3R3BwMIYMGYIlS5ZACAEAGDx4MP744w+DAhEAtm/fjry8PDz99NMAgJkzZ2L58uVYuHAhTp8+jYkTJ2LIkCHYu3ev0XXefvttzJo1C7GxsWjTpg1ycnLQq1cv7Ny5E8eOHUPPnj3x5JNP4vLly4Zjhg0bhmvXrmHPnj3YsGEDfvjhB6Smphqdt3///khNTcWff/6J6OhohIaG4tFHH8WtW7dM3u/69euh0WhM1peXX34Z1tbWWL169R3r9LJly2BlZYXDhw/j888/x7Rp0xAVFVUluS5evIgNGzZg48aNiImJQXJyMgYOHIiRI0ciNjYWe/bswTPPPGP4PRimtnB3d8fWrVuRnZ1tcv/GjRvRpEkTTJs2zdDnAEBBQQHat2+PLVu24NSpUxg9ejSGDh2KI0eOAAC++uordO7cGS+99JLhOG9v7wrnr4m+ddiwYVi9ejW+/vprxMbG4vvvvzeptGIYc0CtVpu01Bw1ahR27dqFxMREw7a4uDjs27cPo0aNAgDY2Nhg6dKlOHPmDL766issWrQIX3755T3Jc6fxMcMw9ymCYe6S4cOHC5lMJqysrIw+FhYWAoBIT08XQgjx008/CTs7O8NxYWFhYty4cUbn6tq1q2jbtq3RuX19fYVWqzVs69+/vxgwYIAQQojExEQhk8nE1atXjc7z6KOPinfeeUcIIURISIj46KOPTMq+e/duIxkrQ19Of28ABADx1FNP3fa4devWCScnJ8P38s9Aj6+vr/jyyy+FEELs2LFDyGQycfnyZcP+06dPCwDiyJEjt70e0zjo0qWLmDdvnhBCCI1GI5ydncXu3buNvi9fvtxQfuDAgYY6U1BQICwtLcXBgweNzjlq1CgxcOBAIUTp+75p06Y7ytKqVSsxf/58IYQQsbGxAoD4999/DfsvXLggABje77///lvY2tqKgoICo/MEBASI77//3uQ1xowZY7Le6GnTpo144oknjGQvX6fDw8PFQw89ZLTtwQcfFFOmTLlruaZOnSoUCoVITU017I+OjhYAREJCQqXyMUxVqKyfKM/evXtFkyZNhEKhEB06dBATJkwQ+/fvNypTtm+5Hb179xaTJ082fA8PDxevv/66UZnydas6fWvZ8547d04AEFFRUXeUj2EaGsOHDxd9+/YVQgih0+lEVFSUUKlU4o033qhQVqvVCi8vLzF16lTDtg8++ED4+PiI4uJik+efPXu2aN++veH71KlTjcbHpupo3759xfDhw4UQdzc+Zhjm/oQtcpgq0aNHD8TExBh9fvzxx9sec+7cOXTs2NFoW/nvANCqVSvIZDLDdw8PD8MK/8mTJ1FcXIygoCBYW1sbPnv37sWlS5cAAK+99ho++eQTdO3aFVOnTsWJEyeqfZ9///03oqOjsXTpUgQFBWHhwoVG+//66y88+uij8PLygo2NDYYOHYqbN28iLy/vrq8RGxsLb29voxXQli1bwt7eHrGxsdWWnbk/OHfuHI4cOYKBAwcCAORyOQYMGGBwR5DL5Xj++eexcuVKAEBubi5+++03DB48GABZlOTl5eGxxx4zqjPLly831Bk9HTp0MPqek5ODN954Ay1atIC9vT2sra0RGxtrsMg5d+4c5HI5QkNDDccEBgbCwcHB8P348ePIycmBk5OT0fXj4+MrXL+madOmjdH3sm3J3crl6+sLFxcXw/e2bdvi0UcfRUhICPr3749FixYhPT29Vu+DaVxcvnzZ6J2cMWMGAKB79+6Ii4vDzp078dxzz+H06dPo1q3bHYMJFxcXY/r06QgJCYGjoyOsra2xfft2I8u6u+Fe+9aYmBjIZDKEh4dX6TiGaShs3rwZ1tbWsLCwwBNPPIEBAwagT58+RvV15cqVkMlkGD58OJYuXQohBHQ6HZYtW4YXXngBUilNudauXYuuXbsaXCnff//9KtfJstzN+JhhmPsTeX0LwJgXVlZWCAwMNNp25cqVGjm3QqEw+i6RSKDT6QDQxFImkyE6OtpI2QPAYJ794osvIjIyElu2bMGOHTswc+ZMzJkzB+PHj6+yLP7+/rC3t0dwcDBSU1MxYMAA7Nu3DwCQkJCAPn36YOzYsfj000/h6OiI/fv3Y9SoUSgqKuJgxkyNsHjxYmi1Wnh6ehq2CSGgUqmwYMEC2NnZYfDgwQgPD0dqaiqioqKgVqvRs2dPADC4XG3ZsgVeXl5G51apVEbfraysjL6/8cYbiIqKwhdffIHAwECo1Wo899xzVQr6m5OTAw8PD5N++qYyugFAUFAQMjMzce3aNaP7Bijg8KVLlwwxOW7HndqSu5Gr/DORyWSIiorCwYMHsWPHDsyfPx/vvfceDh8+DH9//zvKxDB3wtPTEzExMYbvjo6Ohr8VCgW6deuGbt26YcqUKfjkk08wbdo0TJkyxcjttyyzZ8/GV199hXnz5hliuk2YMKHKwbvvtW9Vq9VVuh7DNDR69OiB7777DkqlEp6enpDL5cjPzzeqr25ubgCAkSNHYubMmdi1axd0Oh2SkpLwwgsvAKAslIMHD8bHH3+MyMhI2NnZYc2aNZgzZ06l15ZKpRVceMvG1Lmb8THDMPcnbJHD1DrBwcH4999/jbaV/34nHnjgARQXFyM1NRWBgYFGH3d3d0M5b29vQ4aPyZMnY9GiRQBK49sUFxdXWf5x48bh1KlT+PXXXwEA0dHR0Ol0mDNnDjp16oSgoKAK2YSUSuUdr9WiRQskJSUhKSnJsO3MmTPIyMhAy5Ytqywnc/+g1WqxfPlyzJkzx8j67fjx4/D09DQELO3SpQu8vb2xdu1arFy5Ev379zcoMVq2bAmVSoXLly9XqDOm4mCU5cCBAxgxYgSefvpphISEwN3dHQkJCYb9wcHB0Gq1OHbsmGHbxYsXjSxUQkNDkZKSArlcXuH6zs7OJq/77LPPQqFQmBzULly4ELm5uQYLperW6erIpUcikaBr1674+OOPcezYMSiVSkO7wDD3Svl3sqwipzwtW7aEVqs1pDo21eccOHAAffv2xZAhQ9C2bVs0bdoU58+fNypzN30VcG99a0hICHQ6XYXYXAxjLugXMX18fCCX0xq4Wq02qq/6LG0BAQEIDw/HkiVL8NNPPyEiIgK+vr4AgIMHD8LX1xfvvfceOnTogGbNmhnF0zGFi4uLIe4VQHXt1KlThu93Oz5mGOb+gxU5TK0zfvx4LF68GMuWLcOFCxfwySef4MSJE1VKbRoUFITBgwdj2LBh2LhxI+Lj43HkyBHMnDkTW7ZsAQBMmDAB27dvR3x8PI4ePYrdu3ejRYsWAMhNQiKRYPPmzbhx44ZRgNg7YWlpiZdeeglTp06FEAKBgYHQaDSYP38+4uLisGLFigquV35+fsjJycHOnTuRlpZm0uUqIiICISEhGDx4MI4ePYojR45g2LBhCA8Pr+DqwjQuNm/ejPT0dIwaNQqtW7c2+jz77LNG2V4GDRqEhQsXIioqyuBWBVBAxTfeeAMTJ07EsmXLcOnSJRw9ehTz58/HsmXLbnv9Zs2aGYL8Hj9+HIMGDTJYtABA8+bNERERgdGjR+PIkSM4duwYRo8eDbVabajXERER6Ny5M/r164cdO3YgISEBBw8exHvvvYf//vvP5HV9fHzw+eefY968eXjvvfdw9uxZXLp0CXPnzsVbb72FyZMnIywsDED163R15AKAw4cPY8aMGfjvv/9w+fJlbNy4ETdu3DC0MQxTWzz88MP4/vvvER0djYSEBGzduhXvvvsuevToAVtbWwDU5+zbtw9Xr15FWloaAKrHeiuy2NhYvPzyyxWyIvr5+RkywaWlpRnVcz332rf6+flh+PDhGDlyJDZt2oT4+Hjs2bMHv/zyS00/KoZpEIwaNQobN27Er7/+aghyDFCdvHz5MtasWYNLly7h66+/vuNiwCOPPIItW7Zgy5YtOHv2LMaOHWvIKAfc3fiYYZj7lPoN0cOYE2UDvpWlfLBDUwEcp02bJpydnYW1tbUYOXKkeO2110SnTp1ue+7XX39dhIeHG74XFRWJDz/8UPj5+QmFQiE8PDzE008/LU6cOCGEEOLVV18VAQEBQqVSCRcXFzF06FCRlpZmJIO7u7uQSCSGIHF3uhc9ly9fFnK5XKxdu1YIIcTcuXOFh4eHUKvVIjIyUixfvrzCcWPGjBFOTk4CgCHwXfmAlImJieKpp54SVlZWwsbGRvTv31+kpKSYlI1pPPTp00f06tXL5L7Dhw8LAOL48eNCCCHOnDkjAAhfX1+h0+mMyup0OjFv3jwRHBwsFAqFcHFxEZGRkWLv3r1CiMrf9/j4eNGjRw+hVquFt7e3WLBgQYWAi9euXRNPPPGEUKlUwtfXV6xatUq4urqKhQsXGspkZWWJ8ePHC09PT6FQKIS3t7cYPHiwUYBvU/z222+iW7duhmDq7du3F0uWLKlQzlSdvlNgyLuRq3ywSf1zjoyMFC4uLkKlUomgoCBD8GeGqQ53G+x4xowZonPnzsLR0VFYWFiIpk2bitdee82ofzt06JBo06aNUKlUQj+0u3nzpujbt6+wtrYWrq6u4v333xfDhg0z6mvPnTsnOnXqJNRqtQAg4uPjK7QL1elby9fD/Px8MXHiROHh4SGUSqUIDAw0WacZpqFR2dj3duTl5Qk7Ozvh6OhYIbD+m2++KZycnIS1tbUYMGCA+PLLL43agfL9T1FRkRg7dqxwdHQUrq6uYubMmRX6tDuNjxmGuT+RCMG5U5m657HHHoO7uztWrFhR36IwDFMDXLlyBd7e3oZA4AzDMAzDMAzD1A4c7JipdfLy8rBw4UJERkZCJpNh9erV+OuvvxAVFVXfojEMU0127dqFnJwchISEIDk5GW+99Rb8/PzQvXv3+haNYRiGYRiGYe5rWJHD1DoSiQRbt27Fp59+ioKCAgQHB2PDhg2IiIiob9EYhqkmGo0G7777LuLi4mBjY4MuXbpg5cqVFTJGMQzDMAzDMAxTs7BrFcMwDMMwDMMwDMMwjJnAWasYhmEYhmEYhmEYhmHMBFbkMAzDMAzDMAzDMAzDmAmsyGEYhmEYhmEYhmEYhjETWJHDMAzDMAzDMAzDMAxjJrAih2EYhmEYhmEYhmEYxkxgRQ7DMAzDMAzDMAzDMIyZwIochmEYhmEYhmEYhmEYM4EVOQzDMAzDMAzDMAzDMGYCK3IYhmEYhmEYhmEYhmHMhP8DYyj/MF+JgsQAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAwgAAAKqCAYAAACepnlGAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAACKIElEQVR4nOzdeXhMd/vH8c8kskgiiSCJxJKgltiX2tValGrRKrpYq/TpKrRotbaWrrbSelpq6aMtRXWhVbVUtVpqp3YhtiCC7AnJ+f3h16PTJGTSScbE+3Vd57rMd77nzH3GkNxzfxeLYRiGAAAAAECSi6MDAAAAAHDrIEEAAAAAYCJBAAAAAGAiQQAAAABgIkEAAAAAYCJBAAAAAGAiQQAAAABgIkEAAAAAYCJBAAAAAGAiQQBwW5g3b54sFouOHTtmt2seO3ZMFotF8+bNs9s1nV2rVq3UqlUrR4cBAPgXSBAA5NmRI0c0ePBgVahQQZ6envL19VWzZs00bdo0paSkODo8u/n00081depUR4dhpV+/frJYLPL19c32vT506JAsFossFoveeecdm69/+vRpjR07Vjt27LBDtAAAZ1LE0QEAcE4rVqxQjx495OHhoT59+qhGjRpKT0/Xxo0b9cILL2jv3r368MMPHR2mXXz66afas2ePnn/+eav28uXLKyUlRW5ubg6Jq0iRIkpOTtY333yjhx56yOq5hQsXytPTU6mpqXm69unTpzVu3DiFhYWpTp06uT7vhx9+yNPrAQBuHSQIAGwWFRWlXr16qXz58lq7dq1Kly5tPvfUU0/p8OHDWrFixb9+HcMwlJqaqqJFi2Z5LjU1Ve7u7nJxcVwh1GKxyNPT02Gv7+HhoWbNmumzzz7LkiB8+umn6ty5s5YuXVogsSQnJ8vLy0vu7u4F8noAgPzDECMANnvrrbeUmJioOXPmWCUHf6lUqZKee+458/HVq1c1YcIEVaxYUR4eHgoLC9NLL72ktLQ0q/PCwsJ07733atWqVWrQoIGKFi2q//73v1q/fr0sFos+//xzjR49WqGhofLy8lJ8fLwk6ffff1fHjh3l5+cnLy8vtWzZUr/88stN7+Orr75S586dFRISIg8PD1WsWFETJkxQRkaG2adVq1ZasWKFjh8/bg7ZCQsLk5TzHIS1a9eqRYsW8vb2lr+/v+6//37t27fPqs/YsWNlsVh0+PBh9evXT/7+/vLz81P//v2VnJx809j/8vDDD+u7777TpUuXzLYtW7bo0KFDevjhh7P0j4uL0/Dhw1WzZk35+PjI19dX99xzj3bu3Gn2Wb9+ve68805JUv/+/c37/us+W7VqpRo1amjr1q2666675OXlpZdeesl87u9zEPr27StPT88s99+hQwcVL15cp0+fzvW9AgAKBhUEADb75ptvVKFCBTVt2jRX/R9//HHNnz9fDz74oIYNG6bff/9dkyZN0r59+/Tll19a9T1w4IB69+6twYMHa9CgQapSpYr53IQJE+Tu7q7hw4crLS1N7u7uWrt2re655x7Vr19fY8aMkYuLi+bOnas2bdro559/VsOGDXOMa968efLx8VFkZKR8fHy0du1avfrqq4qPj9fbb78tSXr55Zd1+fJlnTx5UlOmTJEk+fj45HjNH3/8Uffcc48qVKigsWPHKiUlRe+9956aNWumbdu2mcnFXx566CGFh4dr0qRJ2rZtm2bPnq3AwEC9+eabuXpvu3fvriFDhmjZsmUaMGCApGvVg6pVq6pevXpZ+h89elTLly9Xjx49FB4errNnz+q///2vWrZsqT///FMhISGqVq2axo8fr1dffVVPPPGEWrRoIUlWf98XLlzQPffco169eunRRx9VUFBQtvFNmzZNa9euVd++fbVp0ya5urrqv//9r3744Qd98sknCgkJydV9AgAKkAEANrh8+bIhybj//vtz1X/Hjh2GJOPxxx+3ah8+fLghyVi7dq3ZVr58eUOS8f3331v1XbdunSHJqFChgpGcnGy2Z2ZmGnfccYfRoUMHIzMz02xPTk42wsPDjbvvvttsmzt3riHJiIqKsur3T4MHDza8vLyM1NRUs61z585G+fLls/SNiooyJBlz58412+rUqWMEBgYaFy5cMNt27txpuLi4GH369DHbxowZY0gyBgwYYHXNbt26GSVKlMjyWv/Ut29fw9vb2zAMw3jwwQeNtm3bGoZhGBkZGUZwcLAxbtw4M763337bPC81NdXIyMjIch8eHh7G+PHjzbYtW7Zkube/tGzZ0pBkzJo1K9vnWrZsadW2atUqQ5Lx2muvGUePHjV8fHyMrl273vQeAQCOwRAjADb5a1hPsWLFctV/5cqVkqTIyEir9mHDhklSlrkK4eHh6tChQ7bX6tu3r9V8hB07dphDaS5cuKDY2FjFxsYqKSlJbdu21YYNG5SZmZljbH+/VkJCgmJjY9WiRQslJydr//79ubq/vztz5ox27Nihfv36KSAgwGyvVauW7r77bvO9+LshQ4ZYPW7RooUuXLhgvs+58fDDD2v9+vWKiYnR2rVrFRMTk+3wIunavIW/5m1kZGTowoUL8vHxUZUqVbRt27Zcv6aHh4f69++fq77t27fX4MGDNX78eHXv3l2enp7673//m+vXAgAULIYYAbCJr6+vpGu/UOfG8ePH5eLiokqVKlm1BwcHy9/fX8ePH7dqDw8Pz/Fa/3zu0KFDkq4lDjm5fPmyihcvnu1ze/fu1ejRo7V27dosv5Bfvnw5x2vm5K97+fuwqL9Uq1ZNq1atUlJSkry9vc32cuXKWfX7K9aLFy+a7/XNdOrUScWKFdOiRYu0Y8cO3XnnnapUqVK2ez5kZmZq2rRpev/99xUVFWU136JEiRK5ej1JCg0NtWlC8jvvvKOvvvpKO3bs0KeffqrAwMBcnwsAKFgkCABs4uvrq5CQEO3Zs8em8ywWS676ZbdiUU7P/VUdePvtt3NcijOn+QKXLl1Sy5Yt5evrq/Hjx6tixYry9PTUtm3bNGLEiBtWHuzJ1dU123bDMHJ9DQ8PD3Xv3l3z58/X0aNHNXbs2Bz7Tpw4Ua+88ooGDBigCRMmKCAgQC4uLnr++edtuucb/T1lZ/v27Tp37pwkaffu3erdu7dN5wMACg4JAgCb3Xvvvfrwww+1adMmNWnS5IZ9y5cvr8zMTB06dEjVqlUz28+ePatLly6pfPnyeY6jYsWKkq4lLe3atbPp3PXr1+vChQtatmyZ7rrrLrM9KioqS9/cJjd/3cuBAweyPLd//36VLFnSqnpgTw8//LA+/vhjubi4qFevXjn2W7JkiVq3bq05c+ZYtV+6dEklS5Y0H+f2nnMjKSlJ/fv3V0REhJo2baq33npL3bp1M1dKAgDcWpiDAMBmL774ory9vfX444/r7NmzWZ4/cuSIpk2bJuna8BdJWXYinjx5siSpc+fOeY6jfv36qlixot555x0lJiZmef78+fM5nvvXN/d//6Y+PT1d77//fpa+3t7euRpyVLp0adWpU0fz58+3WnZ0z549+uGHH8z3Ij+0bt1aEyZM0IwZMxQcHJxjP1dX1yzViS+++EKnTp2yavsrkfn7feTViBEjFB0drfnz52vy5MkKCwtT3759syxzCwC4NVBBAGCzihUr6tNPP1XPnj1VrVo1q52Uf/31V33xxRfq16+fJKl27drq27evPvzwQ3NYz+bNmzV//nx17dpVrVu3znMcLi4umj17tu655x5Vr15d/fv3V2hoqE6dOqV169bJ19dX33zzTbbnNm3aVMWLF1ffvn317LPPymKx6JNPPsl2aE/9+vW1aNEiRUZG6s4775SPj4+6dOmS7XXffvtt3XPPPWrSpIkGDhxoLnPq5+d3w6E//5aLi4tGjx5903733nuvxo8fr/79+6tp06bavXu3Fi5cqAoVKlj1q1ixovz9/TVr1iwVK1ZM3t7eatSo0Q3niGRn7dq1ev/99zVmzBhz2dW5c+eqVatWeuWVV/TWW2/ZdD0AQAFw7CJKAJzZwYMHjUGDBhlhYWGGu7u7UaxYMaNZs2bGe++9Z7VM6JUrV4xx48YZ4eHhhpubm1G2bFlj1KhRVn0M49oyp507d87yOn8tc/rFF19kG8f27duN7t27GyVKlDA8PDyM8uXLGw899JCxZs0as092y5z+8ssvRuPGjY2iRYsaISEhxosvvmguyblu3TqzX2JiovHwww8b/v7+hiRzydPsljk1DMP48ccfjWbNmhlFixY1fH19jS5duhh//vmnVZ+/ljk9f/68VXt2cWbn78uc5iSnZU6HDRtmlC5d2ihatKjRrFkzY9OmTdkuT/rVV18ZERERRpEiRazus2XLlkb16tWzfc2/Xyc+Pt4oX768Ua9ePePKlStW/YYOHWq4uLgYmzZtuuE9AAAKnsUwbJgJBwAAAKBQYw4CAAAAABMJAgAAAAATCQIAAAAAEwkCAAAAUEA2bNigLl26KCQkRBaLRcuXL7/pOevXr1e9evXk4eGhSpUqad68efkaIwkCAAAAUECSkpJUu3ZtzZw5M1f9o6Ki1LlzZ7Vu3Vo7duzQ888/r8cff1yrVq3KtxhZxQgAAABwAIvFoi+//FJdu3bNsc+IESO0YsUK7dmzx2zr1auXLl26pO+//z5f4qKCAAAAAORRWlqa4uPjrQ577hS/adMmtWvXzqqtQ4cO2rRpk91e459umZ2UV7hVcXQIuE20ntXT0SHgNtF+eVtHh4DbhKePt6NDwG3ix88aODqEbDny98gtL/fWuHHjrNrGjBmjsWPH2uX6MTExCgoKsmoLCgpSfHy8UlJSVLRoUbu8zt/dMgkCAAAA4GxGjRqlyMhIqzYPDw8HRWMfJAgAAABAHnl4eORrQhAcHKyzZ89atZ09e1a+vr75Uj2QSBAAAADg5CxuFkeHkG+aNGmilStXWrWtXr1aTZo0ybfXZJIyAAAAUEASExO1Y8cO7dixQ9K1ZUx37Nih6OhoSdeGLPXp08fsP2TIEB09elQvvvii9u/fr/fff1+LFy/W0KFD8y1GKggAAABwai5FnKeC8Mcff6h169bm47/mL/Tt21fz5s3TmTNnzGRBksLDw7VixQoNHTpU06ZNU5kyZTR79mx16NAh32IkQQAAAAAKSKtWrXSjbciy2yW5VatW2r59ez5GZY0EAQAAAE7N4saoeXvi3QQAAABgIkEAAAAAYGKIEQAAAJyaM01SdgZUEAAAAACYqCAAAADAqRXmjdIcgQoCAAAAABMJAgAAAAATQ4wAAADg1JikbF9UEAAAAACYqCAAAADAqTFJ2b6oIAAAAAAwkSAAAAAAMDHECAAAAE6NScr2RQUBAAAAgIkKAgAAAJyaxZUKgj1RQQAAAABgooIAAAAAp+ZCBcGuqCAAAAAAMJEgAAAAADAxxAgAAABOzeLCECN7ooIAAAAAwEQFAQAAAE7N4sp33vbEuwkAAADARIIAAAAAwJTnIUZbt27Vvn37JEkRERGqV6+e3YICAAAAcot9EOzL5gTh3Llz6tWrl9avXy9/f39J0qVLl9S6dWt9/vnnKlWqlL1jBAAAAFBAbB5i9MwzzyghIUF79+5VXFyc4uLitGfPHsXHx+vZZ5/NjxgBAACAHFlcLA47CiObKwjff/+9fvzxR1WrVs1si4iI0MyZM9W+fXu7BgcAAACgYNmcIGRmZsrNzS1Lu5ubmzIzM+0SFAAAAJBbzEGwL5uHGLVp00bPPfecTp8+bbadOnVKQ4cOVdu2be0aHAAAAICCZXOCMGPGDMXHxyssLEwVK1ZUxYoVFR4ervj4eL333nv5ESMAAACAAmLzEKOyZctq27Zt+vHHH7V//35JUrVq1dSuXTu7BwcAAADcjIUhRnaVp30QLBaL7r77bt199932jgcAAACAA+UqQZg+fbqeeOIJeXp6avr06Tfsy1KnAAAAKEgWF5tHzeMGcpUgTJkyRY888og8PT01ZcqUHPtZLBYSBAAAAMCJ5SpBiIqKyvbPAAAAAAoXm+sx48ePV3Jycpb2lJQUjR8/3i5BAQAAALnFTsr2ZXOCMG7cOCUmJmZpT05O1rhx4+wSFAAAAADHsHkVI8MwZLFkzZZ27typgIAAuwQFAAAA5BY7KdtXrhOE4sWLy2KxyGKxqHLlylZJQkZGhhITEzVkyJB8CRIAAABAwch1gjB16lQZhqEBAwZo3Lhx8vPzM59zd3dXWFiYmjRpki9BAgAAADkprHMBHCXXCULfvn0lSeHh4WratKnc3NzyLSgAAAAAjmHzHISWLVuaf05NTVV6errV876+vv8+KgAAAAAOYXOCkJycrBdffFGLFy/WhQsXsjyfkZFhl8AAAACA3GAnZfuy+d184YUXtHbtWn3wwQfy8PDQ7NmzNW7cOIWEhGjBggX5ESMAAACAAmJzBeGbb77RggUL1KpVK/Xv318tWrRQpUqVVL58eS1cuFCPPPJIfsQJAAAAZItJyvZlcwUhLi5OFSpUkHRtvkFcXJwkqXnz5tqwYYN9owMAAABQoGxOECpUqKCoqChJUtWqVbV48WJJ1yoL/v7+dg0OAAAAQMGyeYhR//79tXPnTrVs2VIjR45Uly5dNGPGDF25ckWTJ0/OjxgBAACAHLGTsn3ZnCAMHTrU/HO7du20f/9+bd26VZUqVVKtWrXsGhwAAACAgmVzgvBP5cuXV/ny5SVJS5Ys0YMPPvivgwIAAAByi0nK9mXTHISrV69qz549OnjwoFX7V199pdq1a7OCEQAAAODkcp0g7NmzR5UqVVLt2rVVrVo1de/eXWfPnlXLli01YMAA3XPPPTpy5Eh+xgoAAABkYXFxcdhRGOV6iNGIESNUqVIlzZgxQ5999pk+++wz7du3TwMHDtT333+vokWL5mecAAAAAApArhOELVu26IcfflCdOnXUokULffbZZ3rppZf02GOP5Wd8AAAAAApQrhOE2NhYhYSESJL8/Pzk7e2txo0b51tgAAAAQG4wSdm+cp0gWCwWJSQkyNPTU4ZhyGKxKCUlRfHx8Vb9fH197R4kAAAAgIKR6wTBMAxVrlzZ6nHdunWtHlssFmVkZNg3QgAAAOAGqCDYV64ThHXr1uVnHAAAAABuAblOEFq2bJmfcQAAAAC4BfzrnZQBAAAAR2KIkX0Vzt0dAAAAAOQJFQQAAAA4tcK6o7GjkCA4mYDmDVRh2ED51ashz5BA/fHAf3T26zWODgtOZtG2Q5q/eb8uJKWqcqC/RrSrpxqlS+TYPyE1XTN+3q21B0/qcmq6Svt6aXibumpRMcTscy4hWdN+2qVfjp5R6tUMlfX30dh7Gqp66YCCuCXcwgY+EqYu7YNVzLuIdu+L1zvvH9LJMyk59u96T2l1vSdEpYM8JUlR0cma9/lx/bY1zuzz3sTaqlvT3+q85d+d1jvvH8qXe4Bz6PtgiDq1KSkf7yLaeyBR0z4+rlMxaTn279KulLrcXUpBJT0kScdPpuiTZae1Zef1Jdw7tympNs1KqFKYl7y9XHX/wO1KSmbFRhRuNicIAwYM0LRp01SsWDGr9qSkJD3zzDP6+OOP7RYcsnL19lL8rgM6MW+pGiyZ6ehw4IRW7YvWu+t26OX29VWjdAl9+sdB/WfxT1r+eCcFeHtm6X8lI0NDFq9XgJen3r6/qQKLeen05SQV83Qz+8SnpqvfwjW6s1ygZvS4S8WLeij6YqJ8Pd0L8tZwC3rkgbJ68N5QvT51v86cTdXjj4Rp8viaevQ/W5R+xcj2nPOx6Zo1P0onT6fIYpHuaRukSS9X14DntyoqOtns9/X3pzV74THzcWpaZn7fDm5hPbsEq1vHQL31wTGdOZ+m/j1C9MbIyhrwwh5dyemzFpeu2Z+d0qmYVElS+7tKavzwShoy6k8dP3mtzcPDRVt2XtaWnZf1eO8yBXY/sI2LK3MQ7Mnmesz8+fOVkpL1m5+UlBQtWLDALkEhZ+dXbdDBMVN19qsfHR0KnNT//jig7rUq6P6aFVSxpJ9e7tBAnm5FtHx3VLb9l++KUnxquiZ3a646ZUopxM9bDcoFqkpgcbPP3N/3KdjXS+M6NVKN0iUU6u+jJuHBKlvcp6BuC7eoHveFasHi49r4+wUdOZak16bsV4kAD7VoXDLHc37ZckG/bY3TyTMpOnE6RR9+ckwpqRmKqGK9EWdqWqbiLl0xj+QUvtW9nXW/J1ALvzyjX7deUlR0it58/5hKFHdTswb+OZ7z27bL2rzjsk7FpOlUTJrmLj6llNRMVat0/f+uZd+d0+dfx2jfoaQCuAvg1pDrCkJ8fLwMw5BhGOaOyn/JyMjQypUrFRgYmC9BArCPKxkZ2hdzUQMaVzPbXCwWNSofpF2nY7M956cjp1UrpKTeWL1V6w+fUnEvD91Trbz6Naoq1/8f8/nT4dNqGhasF776RVtPnFegT1E9VLeSuteuWCD3hVtTSJCnSgZ4aMuOi2ZbUnKG/jwYrxpVfbXm5/M3vYaLi9S6WSl5erpq7/54q+fubhWo9q2DFHcxXb9svqB5i44rjSrCbal0oLtKFHfXtj3XPyNJKRnadyRJEXf4aP2mizc4+xoXi3RX4+Ly9HDRn4cS8zNc4JaX6wTB399fFotFFovFakflv1gsFo0bN86uwQGwr4vJ6cowDAV4WQ8lKuHtqWNx8dmec+pSorZcTtI9EeX13oN36cTFRE1avVVXMzM1uFkNs88XOw7r0TuraGDjCO09E6e31mxXEVcX3VcjPN/vC7emgOLXhphdvHTFqv3ipXTzuZxUKO+tWW/Xlbu7i1JSMvTS63t17MT14UWrfzqnmHOpio1LV8Uwbz3Zr4LKhRbVy5P+tP+N4JZX3O/akMeLl69atV+6fEUB/m7ZnWIKL1tU08dXlbubi1JSMzR28hFFn0rNt1iRP1jm1L5s2knZMAy1adNGS5cuVUDA9YmH7u7uKl++vEJCQm5whevS0tKUlmY9aeiKkSk3CzPQgVtN5v8nFK90aCBXFxdFBAfoXGKKFmzebyYImYYUEVxcz9xVS5JUNai4Dsde1pIdR0gQbiN3twzUC09d/wLpxfG783yt6FPJ6v/cH/LxKqJWzUrp5aFV9MyonWaS8PWqM2bfo8eTdOFiuqa/XlshwZ46HcMvd4Vdm2YBGvp4efPxy2/lfXL6idOpGjzyT3l7uequRsX14pNhihx/gCQBtzWbd1KOiopSuXLlZLHkPVObNGlSlmpDb0uAHnHNeUwqgH+vuJe7XC0WxSVb/+C7kJSqEtlMUJakkt5FVcTVxRxOJEnhJXwVm5SqKxkZcnN1VUkfT1UoYT0+PLyEr9YcPGn/m8Ata+PmC/rz4B/mY3e3a5+Z4v5uunAx3Wwv7u+uw0dvPITj6lVDp85c+5weOJKoancUU4/7QvX2zOx/EfzzwLUKWJnSRUkQbgObtl7S/sPX5wS4uV37naS4XxHF/a1i5e/npiPHkrOc/3dXMwydPnvtS8tDUcmqUsFb3TsGaeqc4/kQOfILy5zal83v5r59+/TLL7+Yj2fOnKk6dero4Ycf1sWLNx/jJ0mjRo3S5cuXrY6HXFgKEchvbq6uqhZcXL8fP2u2ZRqGNh8/q1oh2SfodcqU1ImLCco0rq8CEh2XoJLennJzdb3WJ7Skjl9MsDovOi5BpX298uEucKtKScnQqTOp5hEVnazYuDQ1qH19QrtXUVdFVPbVnv3ZD2nLicUiubnl/CPrjgrXJpX+PRFB4ZWSmqnTZ9PM4/jJVF24mK66Na5/UeFV1EXVKnrbPJ/A4nI94QBuVzYnCC+88ILi46/9x757925FRkaqU6dOioqKUmRkZK6u4eHhIV9fX6uD4UW54+rtJd/aVeVbu6okySu8jHxrV5Vn2dIOjgzO4tEGVfTlzqP6ek+Ujl6I18Qf/lDKlau6v+a1oUCjV/ym6T/tMvv3qFNJ8anpemvNNh2PS9DPR05rzm9/qme9O/52zcraffqC5mz6U9EXE/Tdn8e1dNcR9axbqcDvD7eWL74+pb49y6lZwxKqUN5boyOr6kJcmn7+7fqk+Kmv1VL3zteHqA7uE67a1f0UHOihCuW9NbhPuOrW9NcP689JkkKCPdW3ZzlVqeij4EAPNWtYQqOHVtX2PZd05Bgrzdyuln13To90La0m9f0UXraoRjwZrgsXr+iXPy6Zfd56ubLub1/KfDywV6hqVvVRUEl3hZctqoG9QlW7WjGt+eX6nhvF/YqoYvmiCgm+tldCeNmiqli+qIp5uxbYvQEFzeZ9EKKiohQRESFJWrp0qbp06aKJEydq27Zt6tSpk90DhDW/+jXUZM0n5uOId16SJJ1YsEy7Bo5yVFhwIh2qldPFlDR9sHGPLiSlqkqgv2b2aGkOMYqJT5bL34YQBvt6aWaPlnp37XY9NPd7BRYrqofrV1a/RlXNPtVLl9C7XZvrvQ279OGvexXq560X2tRVp+phBX17uMUsXHpCnp6uevHpyvLxLqLdf17WsDG7rfZACA0uKn/f6xNJi/u5afTQqioR4K6kpKs6cixJkWN264//Xw3p6lVDDeoU10P3lZGnp6vOxaZq/a+xmr+IISG3s0XfxMjTw0VDHw+Tj5er9hxI1Mg3DlrtgRAS5CG/Ytc/a/6+RTTiP+EK8HdTUnKGoqJTNPKNQ9q2+3qFq0u7QPV58HoCO3Xstf/73vogSj9suFAAd4bcYJKyfVkMw8h+95AcBAQEaOPGjYqIiFDz5s3Vp08fPfHEEzp27JgiIiKUnHzjsX45WeFWJU/nAbZqPauno0PAbaL98raODgG3CU8fb0eHgNvEj581cHQI2Tr2+P0Oe+2w2V857LXzi80VhObNmysyMlLNmjXT5s2btWjRIknSwYMHVaYMOwwCAACgYFFBsC+bB/7PmDFDRYoU0ZIlS/TBBx8oNDRUkvTdd9+pY8eOdg8QAAAAQMGxuYJQrlw5ffvtt1nap0yZYpeAAAAAAFuwzKl95endPHLkiEaPHq3evXvr3Llrq0p899132rt3r12DAwAAAFCwbE4QfvrpJ9WsWVO///67li1bpsTEa+sL79y5U2PGjLF7gAAAAAAKjs0JwsiRI/Xaa69p9erVcnd3N9vbtGmj3377za7BAQAAADdjcbE47MiLmTNnKiwsTJ6enmrUqJE2b958w/5Tp05VlSpVVLRoUZUtW1ZDhw5Vamr+7Rpvc4Kwe/dudevWLUt7YGCgYmNjszkDAAAAgCQtWrRIkZGRGjNmjLZt26batWurQ4cO5rD9f/r00081cuRIjRkzRvv27dOcOXO0aNEivfTSS/kWo80Jgr+/v86cOZOlffv27eaKRgAAAEBBsbi4OOyw1eTJkzVo0CD1799fERERmjVrlry8vPTxxx9n2//XX39Vs2bN9PDDDyssLEzt27dX7969b1p1+DdsvqtevXppxIgRiomJkcViUWZmpn755RcNHz5cffr0yY8YAQAAAKeXnp6urVu3ql27dmabi4uL2rVrp02bNmV7TtOmTbV161YzITh69KhWrlypTp065VucNi9zOnHiRD311FMqW7asMjIyFBERoYyMDD388MMaPXp0fsQIAAAA3JLS0tKUlpZm1ebh4SEPD48sfWNjY5WRkaGgoCCr9qCgIO3fvz/b6z/88MOKjY1V8+bNZRiGrl69qiFDhtxaQ4zc3d310Ucf6ciRI/r222/1v//9T/v379cnn3wiV1fX/IgRAAAAyJnF4rBj0qRJ8vPzszomTZpkt1tbv369Jk6cqPfff1/btm3TsmXLtGLFCk2YMMFur/FPNlcQ/lKuXDmVK1fOnrEAAAAATmXUqFGKjIy0asuueiBJJUuWlKurq86ePWvVfvbsWQUHB2d7ziuvvKLHHntMjz/+uCSpZs2aSkpK0hNPPKGXX35ZLvmwSZzNCUJGRobmzZunNWvW6Ny5c8rMzLR6fu3atXYLDgAAALiZvC43ag85DSfKjru7u+rXr681a9aoa9eukqTMzEytWbNGTz/9dLbnJCcnZ0kC/hq1YxhG3gO/AZsThOeee07z5s1T586dVaNGDVksjvsLAQAAAJxJZGSk+vbtqwYNGqhhw4aaOnWqkpKS1L9/f0lSnz59FBoaag5T6tKliyZPnqy6deuqUaNGOnz4sF555RV16dIl34b325wgfP7551q8eHG+zpwGAAAACqOePXvq/PnzevXVVxUTE6M6dero+++/NycuR0dHW1UMRo8eLYvFotGjR+vUqVMqVaqUunTpotdffz3fYrQYNtYmQkJCtH79elWuXNmugaxwq2LX6wE5aT2rp6NDwG2i/fK2jg4BtwlPH29Hh4DbxI+fNXB0CNk6PbS3w147ZMpnDnvt/GLzrIZhw4Zp2rRp+TbmCQAAAIDj5GqIUffu3a0er127Vt99952qV68uNzc3q+eWLVtmv+gAAACAm3DkJOXCKFcJgp+fn9Xjbt265UswAAAAABwrVwnC3Llz8zsOAAAAIE8s+bAXwO2MdxMAAACAyeZlTuvWrZvt3gcWi0Wenp6qVKmS+vXrp9atW9slQAAAAAAFx+YKQseOHXX06FF5e3urdevWat26tXx8fHTkyBHdeeedOnPmjNq1a6evvvoqP+IFAAAArFhcLA47CiObKwixsbEaNmyYXnnlFav21157TcePH9cPP/ygMWPGaMKECbr//vvtFigAAACA/GdzBWHx4sXq3TvrZhS9evXS4sWLJUm9e/fWgQMH/n10AAAAwE1QQbAvmxMET09P/frrr1naf/31V3l6ekqSMjMzzT8DAAAAcB42DzF65plnNGTIEG3dulV33nmnJGnLli2aPXu2XnrpJUnSqlWrVKdOHbsGCgAAACD/2ZwgjB49WuHh4ZoxY4Y++eQTSVKVKlX00Ucf6eGHH5YkDRkyRE8++aR9IwUAAACywz4IdmVzgiBJjzzyiB555JEcny9atGieAwIAAADgOHlKEAAAAIBbRXZ7dCHvcpUgBAQE6ODBgypZsqSKFy9+w7+EuLg4uwUHAAAAoGDlKkGYMmWKihUrJkmaOnVqfsYDAAAA2MTCHAS7ylWC0Ldv32z/DAAAAKBwyfUchPj4+Fz18/X1zXMwAAAAABwr1wmCv7//DeceGIYhi8WijIwMuwQGAAAA5EZh3dHYUXKdIKxbt878s2EY6tSpk2bPnq3Q0NB8CQwAAABAwct1gtCyZUurx66urmrcuLEqVKhg96AAAACAXGOSsl3xbgIAAAAwkSAAAAAAMP2rnZTZtQ4AAACOxiRl+8p1gtC9e3erx6mpqRoyZIi8vb2t2pctW2afyAAAAAAUuFwnCH5+flaPH330UbsHAwAAANjKYmHUvD3lOkGYO3dufsYBAAAA4Bbwr+YgAAAAAA7HHAS7oh4DAAAAwESCAAAAAMDEECMAAAA4NQs7KdsV7yYAAAAAExUEAAAAODU2SrMvKggAAAAATCQIAAAAAEwMMQIAAIBzYydlu+LdBAAAAGCiggAAAACnxiRl+6KCAAAAAMBEBQEAAADOjY3S7Ip3EwAAAICJBAEAAACAiSFGAAAAcGoWC5OU7YkKAgAAAAATFQQAAAA4NyYp2xXvJgAAAAATCQIAAAAAE0OMAAAA4NTYSdm+qCAAAAAAMFFBAAAAgHOz8J23PfFuAgAAADBRQQAAAIBzYw6CXVFBAAAAAGAiQQAAAABgYogRAAAAnJqFScp2xbsJAAAAwHTLVBBaz+rp6BBwm1g3ZJGjQ8BtYsjacY4OAbeJK1eZoInbHJOU7YoKAgAAAAATCQIAAAAA0y0zxAgAAADIC4sL33nbE+8mAAAAABMVBAAAADg3C5OU7YkKAgAAAAATFQQAAAA4N+Yg2BXvJgAAAAATCQIAAAAAE0OMAAAA4NyYpGxXVBAAAAAAmKggAAAAwKmxUZp98W4CAAAAMJEgAAAAADAxxAgAAADOzcJ33vbEuwkAAADARAUBAAAAzs2FZU7tiQoCAAAAABMJAgAAAAATQ4wAAADg1CxMUrYr3k0AAAAAJioIAAAAcG5MUrYrKggAAAAATDZXEFJSUmQYhry8vCRJx48f15dffqmIiAi1b9/e7gECAAAAN8QcBLuy+d28//77tWDBAknSpUuX1KhRI7377ru6//779cEHH9g9QAAAAKAwmTlzpsLCwuTp6alGjRpp8+bNN+x/6dIlPfXUUypdurQ8PDxUuXJlrVy5Mt/iszlB2LZtm1q0aCFJWrJkiYKCgnT8+HEtWLBA06dPt3uAAAAAQGGxaNEiRUZGasyYMdq2bZtq166tDh066Ny5c9n2T09P1913361jx45pyZIlOnDggD766COFhobmW4w2DzFKTk5WsWLFJEk//PCDunfvLhcXFzVu3FjHjx+3e4AAAADADVmcZ5Ly5MmTNWjQIPXv31+SNGvWLK1YsUIff/yxRo4cmaX/xx9/rLi4OP36669yc3OTJIWFheVrjDZXECpVqqTly5frxIkTWrVqlTnv4Ny5c/L19bV7gAAAAMCtKi0tTfHx8VZHWlpatn3T09O1detWtWvXzmxzcXFRu3bttGnTpmzP+frrr9WkSRM99dRTCgoKUo0aNTRx4kRlZGTky/1IeUgQXn31VQ0fPlxhYWFq1KiRmjRpIulaNaFu3bp2DxAAAAC4IRcXhx2TJk2Sn5+f1TFp0qRsw4yNjVVGRoaCgoKs2oOCghQTE5PtOUePHtWSJUuUkZGhlStX6pVXXtG7776r1157ze5v419sHmL04IMPqnnz5jpz5oxq165ttrdt21bdunWza3AAAADArWzUqFGKjIy0avPw8LDb9TMzMxUYGKgPP/xQrq6uql+/vk6dOqW3335bY8aMsdvr/J1NCcKVK1dUtGhR7dixI0u1oGHDhnYNDAAAALjVeXh45DohKFmypFxdXXX27Fmr9rNnzyo4ODjbc0qXLi03Nze5urqabdWqVVNMTIzS09Pl7u6e9+BzYNMQIzc3N5UrVy5fxzwBAAAANrG4OO6wgbu7u+rXr681a9aYbZmZmVqzZo05bP+fmjVrpsOHDyszM9NsO3jwoEqXLp0vyYGUhzkIL7/8sl566SXFxcXlRzwAAABAoRUZGamPPvpI8+fP1759+/Tkk08qKSnJXNWoT58+GjVqlNn/ySefVFxcnJ577jkdPHhQK1as0MSJE/XUU0/lW4w2z0GYMWOGDh8+rJCQEJUvX17e3t5Wz2/bts1uwQEAAAA35eI8y5z27NlT58+f16uvvqqYmBjVqVNH33//vTlxOTo6Wi4u17/DL1u2rFatWqWhQ4eqVq1aCg0N1XPPPacRI0bkW4w2Jwhdu3bNhzAAAACA28PTTz+tp59+Otvn1q9fn6WtSZMm+u233/I5qutsThDya7Y0AAAAkCc2zgXAjeXp3bx06ZJmz56tUaNGmXMRtm3bplOnTtk1OAAAAAAFy+YKwq5du9SuXTv5+fnp2LFjGjRokAICArRs2TJFR0drwYIF+REnAAAAgAJgcwUhMjJS/fr106FDh+Tp6Wm2d+rUSRs2bLBrcAAAAMBNWSyOOwohmxOELVu2aPDgwVnaQ0NDc9wiGgAAAIBzsHmIkYeHh+Lj47O0Hzx4UKVKlbJLUAAAAECuuTBJ2Z5sfjfvu+8+jR8/XleuXJEkWSwWRUdHa8SIEXrggQfsHiAAAACAgmNzgvDuu+8qMTFRgYGBSklJUcuWLVWpUiUVK1ZMr7/+en7ECAAAAKCA2DzEyM/PT6tXr9bGjRu1a9cuJSYmql69emrXrl1+xAcAAADcWCGdLOwoNicIf2nevLmaN29uz1gAAAAAOFieEoQ1a9ZozZo1OnfunDIzM62e+/jjj+0SGAAAAJAr7KRsVzYnCOPGjdP48ePVoEEDlS5dWhZKOgAAAEChYXOCMGvWLM2bN0+PPfZYfsQDAAAA2IZlTu3K5nczPT1dTZs2zY9YAAAAADiYzQnC448/rk8//TQ/YgEAAADgYLkaYhQZGWn+OTMzUx9++KF+/PFH1apVS25ublZ9J0+ebN8IAQAAgBthTqxd5SpB2L59u9XjOnXqSJL27Nlj94AAAAAAOE6uEoR169bldxwAAABA3rDMqV3Z/G4OGDBACQkJWdqTkpI0YMAAuwQFAAAAwDFsThDmz5+vlJSULO0pKSlasGCBXYICAAAA4Bi53gchPj5ehmHIMAwlJCTI09PTfC4jI0MrV65UYGBgvgQJAAAA5IhJynaV6wTB399fFotFFotFlStXzvK8xWLRuHHj7BocAAAAgIKV6wRh3bp1MgxDbdq00dKlSxUQEGA+5+7urvLlyyskJCRfggQAAAByxE7KdpXrBKFly5aSpKioKJUrV04WSjkAAABAoZPrBOEv586d07Rp03Tw4EFJUpUqVdS7d281aNDA7sEBAAAAN2PwxbVd2VSPefHFF9WoUSPNnj1bJ0+e1MmTJ/Xhhx+qUaNGGjFiRH7FCAAAAKCA5DpBmD9/vt577z1Nnz5dFy5c0I4dO7Rjxw7FxcVpypQpmj59OsucAgAAAE4u10OMZs6cqYkTJ+rpp5+2andzc9Ozzz6rq1evasaMGerTp4/dgwQAAAByxE7KdpXrd3Pv3r26//77c3y+a9eu2rt3r12CAgAAAOAYua4guLq6Kj09Pcfnr1y5IldXV7sEBQAAAOQaFQS7yvW7Wa9ePS1cuDDH5z/55BPVq1fPLkEBAAAAcIxcVxCGDx+url27Ki0tTcOGDVNQUJAkKSYmRu+++66mTp2qL7/8Mt8CBQAAAJD/cp0g3HvvvZoyZYqGDx+ud999V35+fpKky5cvq0iRInrnnXd077335lugAAAAQHbYB8G+bNoo7ZlnnlG3bt30xRdf6NChQ5KkypUr64EHHlDZsmXzJUAAAAAABcfmnZTLlCmjoUOH5kcsAAAAgO2YpGxXNicIyD+Lth3S/M37dSEpVZUD/TWiXT3VKF0ix/4Jqema8fNurT14UpdT01Xa10vD29RVi4ohZp9zCcma9tMu/XL0jFKvZqisv4/G3tNQ1UsHFMQtwYkFNG+gCsMGyq9eDXmGBOqPB/6js1+vcXRYuIVtWbtQm1bNUeLlWAWVraqOvUcrtEKtHPv/+cf3Wr98mi7FnlJAUHm1fWC47qjV0nx+wuNVsz2v7YMvqGnHgVZtV6+k6+OJD+nsif0a9OqXCi5XzT43hVvS1vUL9fsPc5QUf16BZarq7p6vKCQ858/a/q3facPX03T5wikFBIapVbfhqljz+mft23kjtec363mU4RHN1fPZOebjJe8P0bkT+5WUcEGeXn4Kq9ZErboNVzH/IPvfIOBgJAi3iFX7ovXuuh16uX191ShdQp/+cVD/WfyTlj/eSQHenln6X8nI0JDF6xXg5am372+qwGJeOn05ScU83cw+8anp6rdwje4sF6gZPe5S8aIeir6YKF9P94K8NTgpV28vxe86oBPzlqrBkpmODge3uL2bV2r14jfU6dGxCq1QW7//OF+fTn1c/3ntO3n7Zv2i48ThbVr24TC16R6pO2q10p7N32rxzKc16NWlCgytLEka+u7PVucc3r1B38wfrWr122e53polb6uYX6DOntifPzeIW8a+P1Zq7ZJJ6vDwOIWE1daWtfO16L2BemLs99l+1k4e2aav5gxTq66Rqliztf7c8o2WznpK/V9aplL//1mTpArVW6hTn0nm4yJFrH9WlqvcWE06DpGPXyklXDqrdUvf0vIPn9NjL36efzeL3GMOgl1Rj7lF/O+PA+peq4Lur1lBFUv66eUODeTpVkTLd0dl23/5rijFp6ZrcrfmqlOmlEL8vNWgXKCqBBY3+8z9fZ+Cfb00rlMj1ShdQqH+PmoSHqyyxX0K6rbgxM6v2qCDY6bq7Fc/OjoUOIHfVs9T3RY9VKf5AyoVUkmdHx0nN3dP7di4NNv+m3/8RJVqNFfTjgNVKqSiWnd9TqXLR2jL2uvLafv4lbI6DuxYq7AqjVS8lPWct8O7N+jI3l/U7qEX8/UecWvY/ONc1W72kGo1fUAlQyqp48Pj5ObmqV2/Zv9Z+2PtAlWo3kKN2j+ukqUr6q77nldwuQhtXf8/q36uRdytPm+e3n5Wzzds10+hFerIr0SoylSsp8YdBulU1A5lZFzJt3sFHIUE4RZwJSND+2IuqlHY9TKli8WiRuWDtOt0bLbn/HTktGqFlNQbq7eq7YzlevDj7zRn05/KyMy83ufwaUUEBeiFr35RmxnL1WveKi3beSTf7wfA7SXjarrOHN+r8IimZpvFxUXh1Zro5NEd2Z5z8ugOhVdratVWoXoznTySff/Ey7E6vPsn1WnxQJb2bxe8oq6Pvyk396zVVhQuGVfTFRO9V2HVrD9rYdWa6tTR7dmec/roDoVVbWLVFh7RXKf+8dmMPrhZ019oog/HdNCqT8coJfFijnGkJF3S3s3fqEyFunJ1dcuxH+Cs8jTE6NKlS1qyZImOHDmiF154QQEBAdq2bZuCgoIUGhpq7xgLvYvJ6cowDAV4Wf9wK+HtqWNx8dmec+pSorZcTtI9EeX13oN36cTFRE1avVVXMzM1uFkNs88XOw7r0TuraGDjCO09E6e31mxXEVcX3VcjPN/vC8DtITnxoozMDPn8Y3iHt29JxcZkXwVNvBybZTiIj29JJV3O/kuRXb8ul7uHt6rVuz68yDAMfT13lOq37KWQsJq6FHvyX94JbnV/fdb++dnxLlZCF2KOZntOYnysvH1LZumfFH/9s1ahegtVqXu3/EqW0aXzJ/TT8sla/N4gPTZikVxcXM1+65a9rW3rF+pKeopCwuuox1Oz7Hh3+Fdc+M7bnmxOEHbt2qV27drJz89Px44d06BBgxQQEKBly5YpOjpaCxYsuOk10tLSlJaWZtWWceWqPNyYEpFbmf+fULzSoYFcXVwUERygc4kpWrB5v5kgZBpSRHBxPXPXtYlbVYOK63DsZS3ZcYQEAYBT2fHLUtVsfK+KuHmYbVvWfKL01CQ16/SEAyNDYRBxZ2fzz4GhVRQYWkWzXmmn6IObraoPjdoPVO1mD+ryhdP6ZcUMfTtvhB586r+yMP4dhYzN6VZkZKT69eunQ4cOydPz+jfenTp10oYNG3J1jUmTJsnPz8/qeGflL7aGUmgU93KXq8WiuORUq/YLSakqkc0EZUkq6V1U5QKKyfVvGXN4CV/FJqXqSkbGtT4+nqpQwtfqvPASvoqJT7bzHQC4nXn5FJfFxVWJ8Res2pPiY+XjVzLbc3z8SirpH/0T42PlnU3/6IN/6EJMlOq06GHVHrX/d508skMTh9TSa09U14yXOkiSZr/2oL6aM+Lf3BJuUX991v752UlKuJClSvAXH9+SVtWCm/WXJP9SZVXUp7gunjv+j9cPUEBQuMIjmum+x6foyJ6fdDpqR95uBnZlWCwOOwojmxOELVu2aPDgwVnaQ0NDFRMTk6trjBo1SpcvX7Y6hndqZmsohYabq6uqBRfX78fPmm2ZhqHNx8+qVkj2/4HVKVNSJy4mKNMwzLbouASV9PaUm+u1cmid0JI6fjHB6rzouASV9vXKh7sAcLtyLeKu0uWr69i+TWabkZmpqP2/qUyFOtmeU6ZCHUX9rb8kRf35q8pUzNp/+8YlKl2+uoLLWi972rH3y3pi7HI9MeZLPTHmS/V+7r+SpAcGT1brbuzXUxi5FnFXcLnqOrbf+rN2fP8mhVaom+05IRXq6Nj+36zaju37VaE5fDYlKf5ijFKSLsnHr1SOfQzj2py/q1fSbbgDwDnYnCB4eHgoPj7ruPiDBw+qVKmc/yH98xq+vr5Wx+0+vOjRBlX05c6j+npPlI5eiNfEH/5QypWrur/mtaFAo1f8puk/7TL796hTSfGp6XprzTYdj0vQz0dOa85vf6pnvTv+ds3K2n36guZs+lPRFxP03Z/HtXTXEfWsW6nA7w/Ox9XbS761q8q39rVfyrzCy8i3dlV5li3t4MhwK2p8dz9t2/CFdv7ypc6fPqKV/xurK2kpqt2suyRp+ZwRWrP0XbN/w3aP6cjejdq06mPFnjmqn756T6eP7dWdbR6xum5aSqL2/bFKdf9RPZAkvxIhCgytbB4lgsIkScVLlZNvQHD+3SwcqmG7/tq5cbF2b/pSsWeOaNVnY5WenqJaTa991r6Z+6LWf3n9s9agTR9F7f1Zv6/+WBdijujnb97TmeN7VL/Vo5Kk9NQkrV36pk4d3aFLsSd1bP8mLf3gPypeqrzCI1pIkk5H7dTWdf/T2RP7dPnCKR3bv0lfz46Uf6lyOSYmgDOz+bfy++67T+PHj9fixYslSRaLRdHR0RoxYoQeeOCBm5yNnHSoVk4XU9L0wcY9upCUqiqB/prZo6U5xCgmPlkufytjBft6aWaPlnp37XY9NPd7BRYrqofrV1a/Rte/YateuoTe7dpc723YpQ9/3atQP2+90KauOlUPK+jbgxPyq19DTdZ8Yj6OeOclSdKJBcu0a+AoR4WFW1T1hp2UnBinn756T4nx5xVUtpoefv4jc4hR/IXTVuO0y1aqp26D3tG6L6dq3ZdTFBAYpoeemmHugfCXvZtXyJCh6g07C5Ckag06KTkhTj9/M/3/N0qrpp7PzDaHDMXHnZHlb7vqlqlYT/cNfEcbvp6qDV9NVvHAMD0wZKa5B4LFxVXnTx3Unt+WKzU5QT5+gQqPaKa77ntORdyu7YVQxN1TB3b8oJ+/fU9X0pLl41dKFaq3UNN7/mP2gYOxk7JdWQzjb2NUcuHy5ct68MEH9ccffyghIUEhISGKiYlRkyZNtHLlSnl7e+cpkOQ5r+bpPMBW64YscnQIuE1cXMumXSgYV64WznHQuPX0b+3oCLKXtGm5w17bu0lXh712frG5guDn56fVq1dr48aN2rVrlxITE1WvXj21a9cuP+IDAAAAbsiggmBXeR7437x5czVv3tyesQAAAABwMJsThOnTp2fbbrFY5OnpqUqVKumuu+6Sq6trtv0AAAAAuyqky406is0JwpQpU3T+/HklJyerePHikqSLFy/Ky8tLPj4+OnfunCpUqKB169apbNmydg8YAAAAQP6xecDWxIkTdeedd+rQoUO6cOGCLly4oIMHD6pRo0aaNm2aoqOjFRwcrKFDWYMaAAAAcDY2VxBGjx6tpUuXqmLFimZbpUqV9M477+iBBx7Q0aNH9dZbb7HkKQAAAAoEk5Tty+Z388yZM7p69WqW9qtXr5o7KYeEhCghISFLHwAAAAC3NpsThNatW2vw4MHavn272bZ9+3Y9+eSTatOmjSRp9+7dCg8Pt1+UAAAAQE4sFscdhZDNCcKcOXMUEBCg+vXry8PDQx4eHmrQoIECAgI0Z84cSZKPj4/efffdm1wJAAAAwK3G5jkIwcHBWr16tfbv36+DBw9KkqpUqaIqVaqYfVq3vkW32QMAAABwQ3neKK1q1aqqWrWqPWMBAAAAbMckZbvKU4Jw8uRJff3114qOjlZ6errVc5MnT7ZLYAAAAAAKns0Jwpo1a3TfffepQoUK2r9/v2rUqKFjx47JMAzVq1cvP2IEAAAAcmQU0snCjmJzPWbUqFEaPny4du/eLU9PTy1dulQnTpxQy5Yt1aNHj/yIEQAAAEABsTlB2Ldvn/r06SNJKlKkiFJSUuTj46Px48frzTfftHuAAAAAAAqOzQmCt7e3Oe+gdOnSOnLkiPlcbGys/SIDAAAAcsPi4rijELJ5DkLjxo21ceNGVatWTZ06ddKwYcO0e/duLVu2TI0bN86PGAEAAAAUEJsThMmTJysxMVGSNG7cOCUmJmrRokW64447WMEIAAAABc4Qk5TtyaYEISMjQydPnlStWrUkXRtuNGvWrHwJDAAAAEDBs2nglKurq9q3b6+LFy/mVzwAAACATQyLi8OOwsjmu6pRo4aOHj2aH7EAAAAAcDCbE4TXXntNw4cP17fffqszZ84oPj7e6gAAAADgvGyepNypUydJ0n333SfL33atMwxDFotFGRkZ9osOAAAAuJlCOtTHUWxOENatW5cfcQAAAAC4BdicILRs2TI/4gAAAADyxLCwzKk95ake8/PPP+vRRx9V06ZNderUKUnSJ598oo0bN9o1OAAAAAAFy+YEYenSperQoYOKFi2qbdu2KS0tTZJ0+fJlTZw40e4BAgAAACg4eVrFaNasWfroo4/k5uZmtjdr1kzbtm2za3AAAADAzbAPgn3ZfFcHDhzQXXfdlaXdz89Ply5dskdMAAAAABzE5gQhODhYhw8fztK+ceNGVahQwS5BAQAAALlmsTjuKIRsThAGDRqk5557Tr///rssFotOnz6thQsXavjw4XryySfzI0YAAAAABcTmZU5HjhypzMxMtW3bVsnJybrrrrvk4eGh4cOH65lnnsmPGAEAAIAcFda5AI5ic4JgsVj08ssv64UXXtDhw4eVmJioiIgI+fj45Ed8AAAAAAqQzenW//73PyUnJ8vd3V0RERFq2LAhyQEAAABQSNicIAwdOlSBgYF6+OGHtXLlSmVkZORHXAAAAECuGLI47MiLmTNnKiwsTJ6enmrUqJE2b96cq/M+//xzWSwWde3aNU+vm1s2Jwhnzpwxg3vooYdUunRpPfXUU/r111/zIz4AAACg0Fi0aJEiIyM1ZswYbdu2TbVr11aHDh107ty5G5537NgxDR8+XC1atMj3GG1OEIoUKaJ7771XCxcu1Llz5zRlyhQdO3ZMrVu3VsWKFfMjRgAAACBHzrRR2uTJkzVo0CD1799fERERmjVrlry8vPTxxx/neE5GRoYeeeQRjRs3rkC2FfhXU769vLzUoUMH3XPPPbrjjjt07NgxO4UFAAAA3PrS0tIUHx9vdaSlpWXbNz09XVu3blW7du3MNhcXF7Vr106bNm3K8TXGjx+vwMBADRw40O7xZydPCUJycrIWLlyoTp06KTQ0VFOnTlW3bt20d+9ee8cHAAAA3LImTZokPz8/q2PSpEnZ9o2NjVVGRoaCgoKs2oOCghQTE5PtORs3btScOXP00Ucf2T32nNi8zGmvXr307bffysvLSw899JBeeeUVNWnSJD9iAwAAAG7OgTsajxo1SpGRkVZtHh4edrl2QkKCHnvsMX300UcqWbKkXa6ZGzYnCK6urlq8eLE6dOggV1dXq+f27NmjGjVq2C04AAAA4Fbm4eGR64SgZMmScnV11dmzZ63az549q+Dg4Cz9jxw5omPHjqlLly5mW2ZmpqRr84IPHDiQL3OAbU4QFi5caPU4ISFBn332mWbPnq2tW7ey7CkAAAAKlPHvptUWGHd3d9WvX19r1qwxlyrNzMzUmjVr9PTTT2fpX7VqVe3evduqbfTo0UpISNC0adNUtmzZfInT5gThLxs2bNCcOXO0dOlShYSEqHv37po5c6Y9YwMAAAAKlcjISPXt21cNGjRQw4YNNXXqVCUlJal///6SpD59+ig0NFSTJk2Sp6dnltE5/v7+kpSvo3ZsShBiYmI0b948zZkzR/Hx8XrooYeUlpam5cuXKyIiIr9iBAAAAHJkOHAOgq169uyp8+fP69VXX1VMTIzq1Kmj77//3py4HB0dLRcXx1ZELIZhGLnp2KVLF23YsEGdO3fWI488oo4dO8rV1VVubm7auXPnv04Qkue8+q/OB3Jr3ZBFjg4Bt4mLa/c7OgTcJq5cdZ5fjuDc+rd2dATZO7tvq8NeO6hafYe9dn7JdQXhu+++07PPPqsnn3xSd9xxR37GBAAAAMBBcl2/2LhxoxISElS/fn01atRIM2bMUGxsbH7GBgAAANyUM+2k7AxyfVeNGzfWRx99pDNnzmjw4MH6/PPPFRISoszMTK1evVoJCQn5GScAAACAAmBz2uPt7a0BAwZo48aN2r17t4YNG6Y33nhDgYGBuu+++/IjRgAAACBHhiwOOwqjf1UXqVKlit566y2dPHlSn332mb1iAgAAAOAgdhk45erqqq5du+rrr7+2x+UAAAAAOEieN0oDAAAAbgWFdbKwo/BuAgAAADBRQQAAAIBTc6adlJ0BFQQAAAAAJioIAAAAcGqFdblRR6GCAAAAAMBEggAAAADAxBAjAAAAODWWObUv3k0AAAAAJioIAAAAcGpMUrYvKggAAAAATCQIAAAAAEwMMQIAAIBTY5KyffFuAgAAADBRQQAAAIBTY5KyfVFBAAAAAGCiggAAAACnxhwE++LdBAAAAGAiQQAAAABgYogRAAAAnBqTlO2LCgIAAAAA0y1TQWi/vK2jQ8BtYsjacY4OAbeJ4m2qOjoE3CZa/Dbd0SHgttHB0QFky7BQQbAnKggAAAAATCQIAAAAAEy3zBAjAAAAIC8MgyFG9kQFAQAAAICJCgIAAACcmsF33nbFuwkAAADARAUBAAAATo2N0uyLCgIAAAAAEwkCAAAAABNDjAAAAODUGGJkX1QQAAAAAJioIAAAAMCpUUGwLyoIAAAAAEwkCAAAAABMDDECAACAU2OIkX1RQQAAAABgooIAAAAAp2YYVBDsiQoCAAAAABMJAgAAAAATQ4wAAADg1JikbF9UEAAAAACYqCAAAADAqVFBsC8qCAAAAABMVBAAAADg1Kgg2BcVBAAAAAAmEgQAAAAAJoYYAQAAwKmxk7J9UUEAAAAAYKKCAAAAAKeWySRlu6KCAAAAAMBEggAAAADAxBAjAAAAODX2QbAvKggAAAAATFQQAAAA4NRY5tS+qCAAAAAAMFFBAAAAgFNjDoJ9UUEAAAAAYCJBAAAAAGBiiBEAAACcGpOU7YsKAgAAAAATFQQAAAA4NSYp2xcVBAAAAAAmEgQAAAAAJoYYAQAAwKkxSdm+bK4gzJ07V1988UWW9i+++ELz58+3S1AAAAAAHMPmBGHSpEkqWbJklvbAwEBNnDjRLkEBAAAAuZXpwKMwsjlBiI6OVnh4eJb28uXLKzo62i5BAQAAAHAMmxOEwMBA7dq1K0v7zp07VaJECbsEBQAAAOSWYVgcdhRGNicIvXv31rPPPqt169YpIyNDGRkZWrt2rZ577jn16tUrP2IEAAAAUEBsXsVowoQJOnbsmNq2basiRa6dnpmZqT59+jAHAQAAAHByNicI7u7uWrRokSZMmKCdO3eqaNGiqlmzpsqXL58f8QEAAAA3xE7K9pXnfRAqV66sypUr2zMWAAAAAA6WqwQhMjJSEyZMkLe3tyIjI2/Yd/LkyXYJDAAAAMgNZ5ssPHPmTL399tuKiYlR7dq19d5776lhw4bZ9v3oo4+0YMEC7dmzR5JUv359TZw4Mcf+9pCrBGH79u26cuWK+WcAAAAAtlu0aJEiIyM1a9YsNWrUSFOnTlWHDh104MABBQYGZum/fv169e7dW02bNpWnp6fefPNNtW/fXnv37lVoaGi+xGgxDMPIlyvbqHmXnxwdAm4TQ168y9Eh4DZRvE1VR4eA20SL36Y7OgTcJnzrd3B0CNn65c9Eh712swgfm/o3atRId955p2bMmCHp2mI/ZcuW1TPPPKORI0fe9PyMjAwVL15cM2bMUJ8+ffIU883YvMzpgAEDlJCQkKU9KSlJAwYMsEtQAAAAQG4ZsjjssEV6erq2bt2qdu3amW0uLi5q166dNm3alKtrJCcn68qVKwoICLDptW1hc4Iwf/58paSkZGlPSUnRggUL7BIUAAAA4AzS0tIUHx9vdaSlpWXbNzY2VhkZGQoKCrJqDwoKUkxMTK5eb8SIEQoJCbFKMuwt1wlCfHy8Ll++LMMwlJCQYPUmXLx4UStXrsx23BQAAACQnzINxx2TJk2Sn5+f1TFp0qR8uc833nhDn3/+ub788kt5enrmy2tINixz6u/vL4vFIovFku3yphaLRePGjbNrcAAAAMCtbNSoUVlW+fTw8Mi2b8mSJeXq6qqzZ89atZ89e1bBwcE3fJ133nlHb7zxhn788UfVqlXr3wV9E7lOENatWyfDMNSmTRstXbrUatyTu7u7ypcvr5CQkHwJEgAAAMiJIzdK8/DwyDEh+Cd3d3fVr19fa9asUdeuXSVdm6S8Zs0aPf300zme99Zbb+n111/XqlWr1KBBA3uEfUO5ThBatmwpSYqKilK5cuVksTjXerMAAACAo0VGRqpv375q0KCBGjZsqKlTpyopKUn9+/eXJPXp00ehoaHmMKU333xTr776qj799FOFhYWZcxV8fHzk42PbCkq5ZfMk5X379umXX34xH8+cOVN16tTRww8/rIsXL9o1OAAAAKAw6dmzp9555x29+uqrqlOnjnbs2KHvv//enLgcHR2tM2fOmP0/+OADpaen68EHH1Tp0qXN45133sm3GG3eB6FmzZp688031alTJ+3evVsNGjTQsGHDtG7dOlWtWlVz587NUyDsg4CCwj4IKCjsg4CCwj4IKCi36j4I6/dkXWGzoLSqUdRhr51fcj3E6C9RUVGKiIiQJC1dulRdunTRxIkTtW3bNnXq1MnuAQIAAAAoODYPMXJ3d1dycrIk6ccff1T79u0lSQEBAYqPj7dvdAAAAMBNGIbjjsLI5gpC8+bNFRkZqWbNmmnz5s1atGiRJOngwYMqU6aM3QMEAAAAUHBsriDMmDFDRYoU0ZIlS/TBBx8oNDRUkvTdd9+pY8eOdg8QAAAAQMGxuYJQrlw5ffvtt1nap0yZYpeAAAAAAFtkOnAfhMIoVwlCfHy8fH19zT/fyF/9AAAAADifXCUIxYsX15kzZxQYGCh/f/9sN0kzDEMWi0UZGRl2DxIAAADIiWFQQbCnXCUIa9euVUBAgCRp3bp1+RrQ7W7gI2Hq0j5YxbyLaPe+eL3z/iGdPJPz2r5d7ymtrveEqHSQpyQpKjpZ8z4/rt+2xpl93ptYW3Vr+ludt/y703rn/UP5cg+4tWxZu1CbVs1R4uVYBZWtqo69Ryu0Qq0c+//5x/dav3yaLsWeUkBQebV9YLjuqNXSfH7C49mv7d/2wRfUtONAq7arV9L18cSHdPbEfg169UsFl6tmn5tCoRLQvIEqDBsov3o15BkSqD8e+I/Ofr3G0WHBySz+YYP+9+1aXbgcrzvKheqFvg+qeqXyNz3vh1+36uUZ89Wyfk29M2yQ2T521v+0YsNmq76Na1XVeyP/Y/fYgVtNrhKEli2v/3IQHh6usmXLZqkiGIahEydO2De628wjD5TVg/eG6vWp+3XmbKoefyRMk8fX1KP/2aL0K9mvo3U+Nl2z5kfp5OkUWSzSPW2DNOnl6hrw/FZFRSeb/b7+/rRmLzxmPk5Ny8zv28EtYO/mlVq9+A11enSsQivU1u8/ztenUx/Xf177Tt6+JbL0P3F4m5Z9OExtukfqjlqttGfzt1o882kNenWpAkMrS5KGvvuz1TmHd2/QN/NHq1r99lmut2bJ2yrmF6izJ/bnzw2iUHD19lL8rgM6MW+pGiyZ6ehw4IR+2LRNU//3pUYO6Kkalcrrs+9+0jNvvK8l745WgF+xHM87ff6Cpn26XHWrVsz2+Sa1q+nVwY+Yj92L2Dx1EwWksC436ig2r2IUHh6u8+fPZ2mPi4tTeHi4XYK6XfW4L1QLFh/Xxt8v6MixJL02Zb9KBHioReOSOZ7zy5YL+m1rnE6eSdGJ0yn68JNjSknNUEQV67kgqWmZirt0xTySUxgKdjv4bfU81W3RQ3WaP6BSIZXU+dFxcnP31I6NS7Ptv/nHT1SpRnM17ThQpUIqqnXX51S6fIS2rF1o9vHxK2V1HNixVmFVGql4qbJW1zq8e4OO7P1F7R56MV/vEc7v/KoNOjhmqs5+9aOjQ4GT+nTlOnVt3VT3tWqsCmVKa9TAh+Tp4a6vf/otx3MyMjP1yswFeuKBTgoJzPqFiXQtISjp72sevj5e+XULwC3F5gThr7kG/5SYmChPT0+7BHU7CgnyVMkAD23ZcdFsS0rO0J8H41Wjau4mfru4SG1blJKnp6v27reeTH53q0B9u7CpFsxooMF9wuXhYfNfPZxMxtV0nTm+V+ERTc02i4uLwqs10cmjO7I95+TRHQqv1tSqrUL1Zjp5JPv+iZdjdXj3T6rT4oEs7d8ueEVdH39Tbu78vwAg/1y5elX7o06oYY0qZpuLi4sa1qii3Yeicjxv9rLvFeBbTPe3bpJjn637Dqv9kJf0wLDX9MacRbqUkGTX2IFbVa5rZZGRkZIki8WiV155RV5e17PojIwM/f7776pTp47dA7xdBBR3lyRdvHTFqv3ipXTzuZxUKO+tWW/Xlbu7i1JSMvTS63t17MT14UWrfzqnmHOpio1LV8Uwbz3Zr4LKhRbVy5P+tP+N4JaRnHhRRmaGfP4xlMjbt6RiY7L/oZl4OTbL0CMf35JKuhybbf9dvy6Xu4e3qtW7PrzIMAx9PXeU6rfspZCwmroUe/Jf3gkA5OxSQpIyMjOzDCUK8CumY6fPZnvOjv1H9PX6TVo4cUSO121aq5pa31lboaVK6OTZWL2/+Bs99+YH+nh8pFxd+JLtVmOwzKld5TpB2L59u6RrP/x3794td/frv7S6u7urdu3aGj58eK6ulZaWprS0NKu2zIx0ubje+BfhwuTuloF64anK5uMXx+/O87WiTyWr/3N/yMeriFo1K6WXh1bRM6N2mknC16vOmH2PHk/ShYvpmv56bYUEe+p0TGrebwK3vR2/LFXNxveqiJuH2bZlzSdKT01Ss05PODAyAMheUkqqxnzwiV56vLf8fX1y7Ne+aX3zz5XKhahSuRB1GzpeW/88ZFWtAAqjXCcIf61e1L9/f02bNu1f7XcwadIkjRs3zqqt7B19Va5K/zxf09ls3HxBfx78w3zs7nbt24ji/m66cDHdbC/u767DRxNveK2rVw2dOnPtF/0DRxJV7Y5i6nFfqN6emf0qRX8euDb8qEzpoiQIhZiXT3FZXFyVGH/Bqj0pPlY+ftnPa/HxK6mkf/RPjI+Vdzb9ow/+oQsxUeo+2HqTxKj9v+vkkR2aOMR6paTZrz2omo3u1f0D38zL7QBAtvyLecvVxUVxlxOs2uMuJ6iEf9YJyifPxur0+TgNe+dDsy3z/2e4Nn70eS1592WVCSqV5bwyQSXlX8xbJ8/GkiDcgjKZpGxXNk/Hnzt37r9+0VGjRplDlv7Ssdfv//q6ziQlJUOn/jFRODYuTQ1qF9fhqGtjHL2Kuiqisq+Wrzxt07UtFsnNLefy5x0Vrn1j8vdEBIWPaxF3lS5fXcf2bVLVuu0kSUZmpqL2/6Y7Wz+S7TllKtRR1L5NanR3X7Mt6s9fVaZinSx9t29cotLlqyu4rPWypx17v6zW3Z4zHydcOqdPpzyuBwZPVmh4bTvcGQBc51akiKqGl9WWvQfV6s5rX0xkZmZqy94D6tH+riz9w0KC9NmbI63aZi1eoaTUNA3r011BJYpn+zpnL1zU5cRklfBnQ1gUfnlar+uPP/7Q4sWLFR0drfR0618yly1bdtPzPTw85OHhYdV2Ow0vyskXX59S357ldOJ0yrVlTh8N04W4NP382/Xx31Nfq6UNm2K1bMW1pGFwn3D9tjVOZ8+nyqtoEd3dMlB1a/orcsy1IUshwZ66u2WgfvsjTpcTrqhimI+efbyitu+5pCPHmGxV2DW+u5+++nikSpevoZDwWtr843xdSUtR7WbdJUnL54xQMf9AtX1gmCSpYbvHtODtPtq06mPdUauV9m5eodPH9qpzn/FW101LSdS+P1bp7oeyjt/1KxFi9djd49p8peKlysk3IDg/bhNOztXbS96VypmPvcLLyLd2VaXHXVbqiTM3OBO45uFOrTVu1v9UrUJZVa9YXp99t14pqenq0rKRJGnM+5+oVICfnu51nzzc3VSprPX/Uz7eRSXJbE9OTdNHS79Tm4a1VcLfVyfPxuq9T79S2aCSalIr+71ggMLE5gTh888/V58+fdShQwf98MMPat++vQ4ePKizZ8+qW7du+RHjbWPh0hPy9HTVi09Xlo93Ee3+87KGjdlttQdCaHBR+fu6mY+L+7lp9NCqKhHgrqSkqzpyLEmRY3brj/9fDenqVUMN6hTXQ/eVkaenq87Fpmr9r7Gav+h4gd8fCl71hp2UnBinn756T4nx5xVUtpoefv4jc4hR/IXTVquSla1UT90GvaN1X07Vui+nKCAwTA89NcPcA+EvezevkCFD1Rt2LtD7QeHkV7+Gmqz5xHwc8c5LkqQTC5Zp18BRjgoLTqR9k3q6FJ+o/y5ZqQuX4lW5fBlNH/mkSvhd+7Y/5sJFWVxyP4nVxcWiw9GnteLnzUpISlGp4n5qVLOqhjzUSe5ubje/AAocOynbl8UwbNtaolatWho8eLCeeuopFStWTDt37lR4eLgGDx6s0qVLZ5lbkFvNu/yUp/MAWw15MWvJGcgPxdvwTSMKRovfpjs6BNwmfOt3cHQI2fpu+5Wbd8on99QtfEmjzet0HTlyRJ07X/vW0N3dXUlJSbJYLBo6dKg+/PDDm5wNAAAA2JdhOO4ojGxOEIoXL66EhGsrBYSGhmrPnj2SpEuXLik5OflGpwIAAAC4xdk8B+Guu+7S6tWrVbNmTfXo0UPPPfec1q5dq9WrV6tt27b5ESMAAACQo0w2SrMrmxOEGTNmKDX12tr5L7/8stzc3PTrr7/qgQce0OjRo+0eIAAAAICCY3OCEBAQYP7ZxcVFI0eOvEFvAAAAAM7E5jkIrq6uOnfuXJb2CxcuyNXV1S5BAQAAALnFJGX7sjlByGlV1LS0NLm7s9kZAAAA4MxyPcRo+vRrayxbLBbNnj1bPj4+5nMZGRnasGGDqlZlzW8AAAAULDZKs69cJwhTpkyRdK2CMGvWLKvhRO7u7goLC9OsWbPsHyEAAACAApPrBCEqKkqS1Lp1ay1btkzFixfPt6AAAAAAOIbNqxitW7fO6vHVq1eVmppqNeQIAAAAKCiZhXSysKPkepLyN998o3nz5lm1vf766/Lx8ZG/v7/at2+vixcv2js+AAAAAAUo1wnC5MmTlZSUZD7+9ddf9eqrr+qVV17R4sWLdeLECU2YMCFfggQAAABywjKn9pXrBGHv3r1q2rSp+XjJkiW6++679fLLL6t79+5699139c033+RLkAAAAAAKRq4ThISEBJUoUcJ8vHHjRrVt29Z8XL16dZ0+fdq+0QEAAAAoULlOEEJDQ7Vv3z5JUmJionbu3GlVUbhw4YK8vLzsHyEAAABwA4YsDjsKo1wnCD169NDzzz+vTz75RIMGDVJwcLAaN25sPv/HH3+oSpUq+RIkAAAAgIKR62VOX331VZ06dUrPPvusgoOD9b///c9qs7TPPvtMXbp0yZcgAQAAgJywzKl95TpBKFq0qBYsWJDj8//cHwEAAACA88n1EKPsvPHGG7p06ZKdQgEAAABsxzKn9vWvEoSJEycqLi7OXrEAAAAAcLB/lSAYhTVtAgAAAG5TuU4Qxo8fr+Tk5PyMBQAAALAZQ4zsK9cJwrhx45SYmGjV9ueff6p8+fJ2DwoAAACAY+R6FaPshhOVLVvWrsEAAAAAtso0CueGZY5i0xwEi4U3HwAAACjMcl1BkKTKlSvfNElgVSMAAADAedmUIIwbN05+fn75FQsAAABgs8I6WdhRbEoQevXqpcDAwPyKBQAAAICD5TpBYP4BAAAAbkVUEOwr15OU2RQNAAAAKPxyXUHIzMzMzzgAAACAPMnke2y7smmZUwAAAACFGwkCAAAAAJNNqxgBAAAAtxqDnZTtigoCAAAAABMVBAAAADg1Ftu0LyoIAAAAAEwkCAAAAABMDDECAACAU2MfBPuiggAAAADARAUBAAAATo1JyvZFBQEAAACAiQoCAAAAnBoVBPuiggAAAADARIIAAAAAwMQQIwAAADg1ljm1LyoIAAAAAExUEAAAAODUmKRsX1QQAAAAAJhIEAAAAACYGGIEAAAAp5aZ6egIChcqCAAAAABMVBAAAADg1JikbF9UEAAAAACYqCAAAADAqVFBsC8qCAAAAABMJAgAAAAATAwxAgAAgFPLZIiRXVFBAAAAAGCiggAAAACnZjh0lrLFga+dP6ggAAAAADCRIAAAAAAwkSAAAADAqRmG4468mDlzpsLCwuTp6alGjRpp8+bNN+z/xRdfqGrVqvL09FTNmjW1cuXKvL1wLpEgAAAAAAVk0aJFioyM1JgxY7Rt2zbVrl1bHTp00Llz57Lt/+uvv6p3794aOHCgtm/frq5du6pr167as2dPvsVIggAAAACnlpnpuMNWkydP1qBBg9S/f39FRERo1qxZ8vLy0scff5xt/2nTpqljx4564YUXVK1aNU2YMEH16tXTjBkz/uW7ljMSBAAAACCP0tLSFB8fb3WkpaVl2zc9PV1bt25Vu3btzDYXFxe1a9dOmzZtyvacTZs2WfWXpA4dOuTY3x5IEAAAAODUHDkHYdKkSfLz87M6Jk2alG2csbGxysjIUFBQkFV7UFCQYmJisj0nJibGpv72wD4IAAAAQB6NGjVKkZGRVm0eHh4OisY+SBAAAACAPPLw8Mh1QlCyZEm5urrq7NmzVu1nz55VcHBwtucEBwfb1N8eGGIEAAAAp5ZpOO6whbu7u+rXr681a9Zcjz0zU2vWrFGTJk2yPadJkyZW/SVp9erVOfa3ByoIAAAAQAGJjIxU37591aBBAzVs2FBTp05VUlKS+vfvL0nq06ePQkNDzXkMzz33nFq2bKl3331XnTt31ueff64//vhDH374Yb7FeMskCJ4+3o4OAbeJK1ctjg4Bt4kWv013dAi4Tfzc+FlHh4DbROcrBxwdQrbyumGZI/Ts2VPnz5/Xq6++qpiYGNWpU0fff/+9ORE5OjpaLi7XB/k0bdpUn376qUaPHq2XXnpJd9xxh5YvX64aNWrkW4y3TIIAAAAA3A6efvppPf3009k+t379+ixtPXr0UI8ePfI5quuYgwAAAADARAUBAAAATs2wdbawXRW+octUEAAAAACYqCAAAADAqTm0gFAIUUEAAAAAYKKCAAAAAKfmTMucOgMqCAAAAABMJAgAAAAATAwxAgAAgFPLZJayXVFBAAAAAGCiggAAAACnxiRl+6KCAAAAAMBEggAAAADAxBAjAAAAODWGGNkXFQQAAAAAJioIAAAAcGqZlBDsigoCAAAAABMJAgAAAAATQ4wAAADg1IxMR0dQuFBBAAAAAGCiggAAAACnZjBJ2a6oIAAAAAAwUUEAAACAU8tkDoJdUUEAAAAAYCJBAAAAAGBiiBEAAACcGpOU7YsKAgAAAAATFQQAAAA4tUwKCHZFBQEAAACAiQQBAAAAgIkhRgAAAHBqBmOM7IoKAgAAAAATFQQAAAA4NVY5tS8qCAAAAABMVBAAAADg1DKZg2BXVBAAAAAAmEgQAAAAAJgYYgQAAACnZjBL2a6oIAAAAAAwUUEAAACAUzMyHR1B4UIFAQAAAICJBAEAAACAiSFGAAAAcGqZTFK2KyoIAAAAAExUEAAAAODUWObUvqggAAAAADBRQQAAAIBTy8ykgmBPVBAAAAAAmEgQAAAAAJgYYgQAAACnxhxl+6KCAAAAAMBEBQEAAABOzWCSsl1RQQAAAABgIkEAAAAAYMpzgnDkyBGNHj1avXv31rlz5yRJ3333nfbu3Wu34AAAAICbyTQMhx2FUZ4ShJ9++kk1a9bU77//rmXLlikxMVGStHPnTo0ZM8auAQIAAAAoOHlKEEaOHKnXXntNq1evlru7u9nepk0b/fbbb3YLDgAAALgZI9Nw2FEY5SlB2L17t7p165alPTAwULGxsf86KAAAAACOkadlTv39/XXmzBmFh4dbtW/fvl2hoaF2CQwAAADIjcL6Tb6j5KmC0KtXL40YMUIxMTGyWCzKzMzUL7/8ouHDh6tPnz72jhEAAABAAclTgjBx4kRVrVpVZcuWVWJioiIiInTXXXepadOmGj16tL1jBAAAAFBA8jTEyN3dXR999JFeeeUV7dmzR4mJiapbt67uuOMOe8cHAAAA3BAjjOwrTwnCX8qVK6dy5crZKxYAAAAADpanBGHAgAE3fP7jjz/OUzAAAACArZikbF95ShAuXrxo9fjKlSvas2ePLl26pDZt2tglMAAAAAAFL08JwpdffpmlLTMzU08++aQqVqz4r4MCAAAA4Bh5WsUo2wu5uCgyMlJTpkyx1yUBAACAmzIMw2FHYWS3BEGSjhw5oqtXr9rzkgAAAAAKUJ6GGEVGRlo9NgxDZ86c0YoVK9S3b1+7BAYAAADkRiaTlO0qTwnC9u3brR67uLioVKlSevfdd2+6whFurO+DIerUpqR8vIto74FETfv4uE7FpOXYv0u7UupydykFlfSQJB0/maJPlp3Wlp3xZp/ObUqqTbMSqhTmJW8vV90/cLuSkjPy/V5wa9i6fqF+/2GOkuLPK7BMVd3d8xWFhNfKsf/+rd9pw9fTdPnCKQUEhqlVt+GqWLOl+fy380Zqz2/W85DCI5qr57NzzMdL3h+icyf2Kynhgjy9/BRWrYladRuuYv5B9r9B3NIW/7BB//t2rS5cjtcd5UL1Qt8HVb1S+Zue98OvW/XyjPlqWb+m3hk2yGwfO+t/WrFhs1XfxrWq6r2R/7F77Ch8Apo3UIVhA+VXr4Y8QwL1xwP/0dmv1zg6LOCWk6cEYd26dfaOA5J6dglWt46BeuuDYzpzPk39e4TojZGVNeCFPbpyJfvM+HxcumZ/dkqnYlIlSe3vKqnxwytpyKg/dfzktTYPDxdt2XlZW3Ze1uO9yxTY/cDx9v2xUmuXTFKHh8cpJKy2tqydr0XvDdQTY7+Xt2+JLP1PHtmmr+YMU6uukapYs7X+3PKNls56Sv1fWqZSoZXNfhWqt1CnPpPMx0WKuFtdp1zlxmrScYh8/Eop4dJZrVv6lpZ/+Jwee/Hz/LtZ3HJ+2LRNU//3pUYO6Kkalcrrs+9+0jNvvK8l745WgF+xHM87ff6Cpn26XHWrZr/oRZPa1fTq4EfMx+5F/tWWPriNuHp7KX7XAZ2Yt1QNlsx0dDiwo8I6F8BR7DoHAf9O93sCtfDLM/p16yVFRafozfePqURxNzVr4J/jOb9tu6zNOy7rVEyaTsWkae7iU0pJzVS1Sj5mn2XfndPnX8do36GkArgL3Eo2/zhXtZs9pFpNH1DJkErq+PA4ubl5atevS7Pt/8faBapQvYUatX9cJUtX1F33Pa/gchHauv5/Vv1ci7jLx6+UeXh6+1k937BdP4VWqCO/EqEqU7GeGncYpFNRO5SRcSXf7hW3nk9XrlPX1k11X6vGqlCmtEYNfEieHu76+qffcjwnIzNTr8xcoCce6KSQwKxJrHQtISjp72sevj5e+XULKGTOr9qgg2Om6uxXPzo6FOCWluuvXerWrSuLxZKrvtu2bctzQLer0oHuKlHcXdv2XB8alJSSoX1HkhRxh4/Wb7p4g7OvcbFIdzUuLk8PF/15KDE/w4UTyLiarpjovWrScbDZZnFxUVi1pjp1dHu255w+ukN3tutn1RYe0VwHd1j/MI0+uFnTX2giTy9fla/SWHfd97yK+hTP9popSZe0d/M3KlOhrlxd3f7dTcFpXLl6VfujTqjffXebbS4uLmpYo4p2H4rK8bzZy75XgG8x3d+6ibYfOJJtn637Dqv9kJdUzNtLd0bcoSEP3Sv/Yt52vwcAuF3lOkHo2rVrPoaB4n7XfnG6eNl6FahLl68owP/Gv1SFly2q6eOryt3NRSmpGRo7+YiiT6XmW6xwDsmJF2VkZmQZSuRdrIQuxBzN9pzE+Fh5+5bM0j8pPtZ8XKF6C1Wpe7f8SpbRpfMn9NPyyVr83iA9NmKRXFxczX7rlr2tbesX6kp6ikLC66jHU7PseHe41V1KSFJGZmaWoUQBfsV07PTZbM/Zsf+Ivl6/SQsnjsjxuk1rVVPrO2srtFQJnTwbq/cXf6Pn3vxAH4+PlKsLRXHgdsVOyvaV6wRhzJgxdnvRtLQ0paVZT7zNzEiXi6t7DmcUPm2aBWjo49cn6r381qE8X+vE6VQNHvmnvL1cdVej4nrxyTBFjj9AkoB8EXFnZ/PPgaFVFBhaRbNeaafog5sVVrWJ+Vyj9gNVu9mDunzhtH5ZMUPfzhuhB5/6b64rkbi9JKWkaswHn+ilx3vL39cnx37tm9Y3/1ypXIgqlQtRt6HjtfXPQ2pYo0pBhAoAhZ5DZnZNmjRJ48aNs2oLrz5IFWo+4YhwHGLT1kvaf/j6nAA3t2u/NBX3K6K4S9fHafv7uenIseQbXutqhqHTZ68lXIeiklWlgre6dwzS1DnH8yFyOAsvn+KyuLgqKf6CVXtSwoUsVYK/+PiWtKoW3Ky/JPmXKquiPsV18dxxqwTByydAXj4BCggKV4nSFfX+qJY6HbVDoRXq/ou7grPwL+YtVxcXxV1OsGqPu5ygEv5ZJyifPBur0+fjNOydD822zP+fdNj40ee15N2XVSaoVJbzygSVlH8xb508G0uCANzGqCDYV54ShIyMDE2ZMkWLFy9WdHS00tPTrZ6Pi4u74fmjRo3KspdC18f35CUUp5WSmqmUVOsqyoWL6apbw1dHjqdIkryKuqhaRW99s/qcTde2uFxPOHD7ci3iruBy1XVs/yZVrtNOkmRkZur4/k2q1+rRbM8JqVBHx/b/pjvb9jPbju37VaEV6uT4OvEXY5SSdEk+fll/efuLYWRKkq5eSc+xDwoXtyJFVDW8rLbsPahWd15bVjczM1Nb9h5Qj/Z3ZekfFhKkz94cadU2a/EKJaWmaVif7goqkf0cl7MXLupyYrJK+Pva/yYA4DaVpwRh3Lhxmj17toYNG6bRo0fr5Zdf1rFjx7R8+XK9+uqrNz3fw8NDHh4eVm230/CinCz77pwe6Vpap2JSFXMuXf16hOjCxSv65Y9LZp+3Xq6sX7Zc1Fc/nJckDewVqs07LutcbLq8irqqTbMA1a5WTCPfuD5kqbhfEQX4uykk+Np7Hl62qFJSM3QuNl0JSeyHUJg1bNdf384bodLla6h0WC39sXa+0tNTVKtpd0nSN3NfVDH/ILXqNkyS1KBNH3367mP6ffXHqlSzpf7cslJnju9Rx0fGS5LSU5O0ccUMVanbQd6+JXUp9oTWLXtbxUuVV3hEC0nS6aidOnNst8pUqi9PL19dPB+tn7+eJv9S5age3GYe7tRa42b9T9UqlFX1iuX12XfrlZKari4tG0mSxrz/iUoF+OnpXvfJw91NlcqGWJ3v411Uksz25NQ0fbT0O7VpWFsl/H118mys3vv0K5UNKqkmtaoW7M3BKbl6e8m7UjnzsVd4GfnWrqr0uMtKPXHGgZEBt5Y8JQgLFy7URx99pM6dO2vs2LHq3bu3KlasqFq1aum3337Ts88+a+84bwuLvomRp4eLhj4eJh8vV+05kKiRbxy02gMhJMhDfsWuT1r29y2iEf8JV4C/m5KSMxQVnaKRbxzStt3XV0Pq0i5QfR68/oN36thrP0jf+iBKP2ywHn6CwqVag05KTojTz99M//+N0qqp5zOzzSFD8XFnZLFcn9hZpmI93TfwHW34eqo2fDVZxQPD9MCQmeYeCBYXV50/dVB7fluu1OQE+fgFKjyime667zkVcbuW5Bdx99SBHT/o52/f05W0ZPn4lVKF6i3U9J7/mH1we2jfpJ4uxSfqv0tW6sKleFUuX0bTRz6pEn7Xvu2PuXBRFpfcVztdXCw6HH1aK37erISkFJUq7qdGNatqyEOd5O7GClm4Ob/6NdRkzSfm44h3XpIknViwTLsGjnJUWLCDTPZBsCuLkYedJby9vbVv3z6VK1dOpUuX1ooVK1SvXj0dPXpUdevW1eXLl20OpF3vP2w+B8iLR55o4OgQcJt4wHeVo0PAbeLnxnwxh4LR+coBR4eQrX5js18drSDMGxvksNfOL3laE65MmTI6c+ZaKa5ixYr64YcfJElbtmzJMnQIAAAAyE9GpuGwI7/ExcXpkUceka+vr/z9/TVw4EAlJua8z1VcXJyeeeYZValSRUWLFlW5cuX07LPP5umL+zwlCN26ddOaNWskSc8884xeeeUV3XHHHerTp48GDBiQl0sCAAAA+H+PPPKI9u7dq9WrV+vbb7/Vhg0b9MQTOa/4efr0aZ0+fVrvvPOO9uzZo3nz5un777/XwIEDbX5tm4YYzZgxQ48++qj8/f2t2jdt2qRNmzbpjjvuUJcuXWwOQmKIEQoOQ4xQUBhihILCECMUlFt1iFGfVxw3yXzBhNJ2v+a+ffsUERGhLVu2qEGDa7+3fP/99+rUqZNOnjypkJCQm1zhmi+++EKPPvqokpKSVKRI7qce21RBePnllxUSEqJHHnlEa9euNdubNGmiyMjIPCcHAAAAgDNKS0tTfHy81fHPDYFttWnTJvn7+5vJgSS1a9dOLi4u+v3333N9ncuXL8vX19em5ECyMUGIiYnRrFmzdPr0ad19990KDw/XhAkTdOLECZteFAAAACgMJk2aJD8/P6tj0qRJ/+qaMTExCgwMtGorUqSIAgICFBMTk6trxMbGasKECTcclpQTmxKEokWLqk+fPlq3bp0OHTqkxx57THPmzFF4eLg6duyoL774QleuXLn5hQAAAAA7ycw0HHaMGjVKly9ftjpGjcp+2dyRI0fKYrHc8Ni/f/+/fj/i4+PVuXNnRUREaOzYsTafn6d9ECSpQoUKGj9+vMaNG6cff/xR8+bNU79+/eTt7a1z52zb+RcAAABwRtltAJyTYcOGqV+/fjfsU6FCBQUHB2f5ffrq1auKi4tTcHDwDc9PSEhQx44dVaxYMX355Zdyy8M+MXlOEP5isVhUpEgRWSwWGYZBBQEAAAAFKj+XG7WnUqVKqVSpUjft16RJE126dElbt25V/fr1JUlr165VZmamGjVqlON58fHx6tChgzw8PPT111/L09MzT3HmaZlTSTpx4oTGjx+vChUq6O6779bp06f10UcfmfsjAAAAALBdtWrV1LFjRw0aNEibN2/WL7/8oqefflq9evUyVzA6deqUqlatqs2bN0u6lhy0b99eSUlJmjNnjuLj4xUTE6OYmBhlZGTY9Po2VRDS09O1bNkyffzxx1q7dq1Kly6tvn37asCAAapQoYJNLwwAAAAgewsXLtTTTz+ttm3bysXFRQ888ICmT59uPn/lyhUdOHBAycnJkqRt27aZKxxVqlTJ6lpRUVEKCwvL9WvblCAEBwcrOTlZ9957r7755ht16NBBLi55LkIAAAAA/5oN23o5jYCAAH366ac5Ph8WFmZ1361atbLb+2DTb/ejR4/WiRMntGTJEt1zzz1ycXHRZ599pqSkJLsEAwAAAMCxbKogREZGZmkbPHiwGjVqxBAjAAAAOISRmenoEAqVfz0+qDCWdAAAAIDbFRMIAAAAAJhsGmKUmZmpt99+W19//bXS09PVtm1brVy5UqGhofkVHwAAAHBDmU6yD4KzsKmC8Prrr+ull16Sj4+PQkNDNW3aNM2dOzfXu8cBAAAAuLXZVEFYsGCB3n//fQ0ePFiS9OOPP6pz586aPXs2y50CAADAIZgTa182/VYfHR2tTp06mY/btWsni8Wi06dP2z0wAAAAAAXPpgrC1atX5enpadXm5uamK1eu2DUoAAAAILcM5iDYlU0JgmEY6tevn9Wcg9TUVA0ZMkTe3t5m27Jly+wXIQAAAIACY1OC0Ldv3yxtjz76qN2CAQAAAOBYNiUIc+fOza84AAAAgDxhiJF9sfQQAAAAAJNNFQQAAADgVpNpZDo6hEKFCgIAAAAAEwkCAAAAABNDjAAAAODUmKRsX1QQAAAAAJioIAAAAMCpUUGwLyoIAAAAAExUEAAAAODUDIMKgj1RQQAAAABgIkEAAAAAYGKIEQAAAJxaZiY7KdsTFQQAAAAAJioIAAAAcGosc2pfVBAAAAAAmEgQAAAAAJgYYgQAAACnZhhMUrYnKggAAAAATFQQAAAA4NSYpGxfVBAAAAAAmKggAAAAwKlRQbAvKggAAAAATCQIAAAAAEwMMQIAAIBTy2SZU7uiggAAAADARAUBAAAATo1JyvZFBQEAAACAiQQBAAAAgIkhRgAAAHBqRiaTlO2JCgIAAAAAExUEAAAAODUmKdsXFQQAAAAAJioIAAAAcGoGG6XZFRUEAAAAACYSBAAAAAAmhhgBAADAqWUySdmuqCAAAAAAMFFBAAAAgFNjozT7ooIAAAAAwESCAAAAAMDEECMAAAA4NXZSti8qCAAAAABMVBAAAADg1NhJ2b6oIAAAAAAwUUEAAACAU2MOgn1RQQAAAABgIkEAAAAAYGKIEQAAAJwaOynbFxUEAAAAACaLYRjM6nBCaWlpmjRpkkaNGiUPDw9Hh4NCjM8aCgqfNRQUPmvAjZEgOKn4+Hj5+fnp8uXL8vX1dXQ4KMT4rKGg8FlDQeGzBtwYQ4wAAAAAmEgQAAAAAJhIEAAAAACYSBCclIeHh8aMGcPkKuQ7PmsoKHzWUFD4rAE3xiRlAAAAACYqCAAAAABMJAgAAAAATCQIAAAAAEwkCDaaN2+e/P39bTqnX79+6tq1a77EcysJCwvT1KlTHR0GnFSrVq30/PPPOzoM3MLWr18vi8WiS5cu5fkafM6Qk7Fjx6pOnTqODgO4JZAg/L+cfon/5w+knj176uDBgwUb3E3k9ofmX/3+OkqVKqVOnTpp9+7dNr1eTknSli1b9MQTT9h0rcJs06ZNcnV1VefOnR0dSoH59ttv1bJlSxUrVkxeXl668847NW/ePKs+9vglD/b39/8bsjvGjh2b5Zzk5GSNGjVKFStWlKenp0qVKqWWLVvqq6++Mvvk9YuD7H6Rb9q0qc6cOSM/P7+bnp/T52zZsmWaMGGCzfGg4PXr18/8/Lm7u6tSpUoaP368rl69atXv3XffVfHixZWamprlGsnJyfL19dX06dMLKmygUCBBsFHRokUVGBjo6DD+lQMHDujMmTNatWqV0tLS1LlzZ6Wnp//r65YqVUpeXl52iLBwmDNnjp555hlt2LBBp0+fztfXMgwjyw/Ngvbee+/p/vvvV7NmzfT7779r165d6tWrl4YMGaLhw4c7JCZ7fK5vF2fOnDGPqVOnytfX16otu7/DIUOGaNmyZXrvvfe0f/9+ff/993rwwQd14cKFfInR3d1dwcHBslgseb5GQECAihUrZseokJ86duyoM2fO6NChQxo2bJjGjh2rt99+26rPY489pqSkJC1btizL+UuWLFF6eroeffTRggoZKBwMGIZhGH379jXuv//+LO3r1q0zJBkXL140DMMw5s6da/j5+Vn1mTBhglGqVCnDx8fHGDhwoDFixAijdu3aWa799ttvG8HBwUZAQIDxn//8x0hPTzf7pKamGsOGDTNCQkIMLy8vo2HDhsa6devM548dO2bce++9hr+/v+Hl5WVEREQYK1asMKKiogxJVkffvn2zvcd/3othGMbXX39tSDJ27txptr377rtGjRo1DC8vL6NMmTLGk08+aSQkJFhd4+/HmDFjDMMwjPLlyxtTpkwxr3P8+HHjvvvuM7y9vY1ixYoZPXr0MGJiYnL8OyhMEhISDB8fH2P//v1Gz549jddff918rnfv3sZDDz1k1T89Pd0oUaKEMX/+fMMwDCMjI8OYOHGiERYWZnh6ehq1atUyvvjiC7P/X38PK1euNOrVq2e4ubkZ69atMw4fPmzcd999RmBgoOHt7W00aNDAWL16tdVrnT592ujUqZPh6elphIWFGQsXLszyd3fx4kVj4MCBRsmSJY1ixYoZrVu3Nnbs2JHj/UZHRxtubm5GZGRkluemT59uSDJ+++23G35eW7ZsaTzzzDPGCy+8YBQvXtwICgoyP1u5jWvMmDFG7dq1jY8++sgICwszLBaLYRiG8cUXXxg1atQwPD09jYCAAKNt27ZGYmJijvdzu8vu/7ns+Pn5GfPmzcvx+ZYtW2b5+zYMw4iNjTV69eplhISEGEWLFjVq1KhhfPrpp+Z5ffv2zXJeVFRUlv/D8vL/YsuWLY3nnnvOfK3U1FTjxRdfNMqUKWO4u7sbFStWNGbPnm3zewb7y+7n8t133200btw4S9/u3bsbbdu2zdLesmVLo2fPnoZhGMaLL75o3HHHHUbRokWN8PBwY/To0VY/h//6/+Pv5/79s2IYhnH//fdb/Yy92c9uwFlRQfiXFi5cqNdff11vvvmmtm7dqnLlyumDDz7I0m/dunU6cuSI1q1bp/nz52vevHlWQy+efvppbdq0SZ9//rl27dqlHj16qGPHjjp06JAk6amnnlJaWpo2bNig3bt3680335SPj4/Kli2rpUuXSrpeGZg2bVquYr98+bI+//xzSde+mfuLi4uLpk+frr1792r+/Plau3atXnzxRUnXSvz//HYxu28WMzMzdf/99ysuLk4//fSTVq9eraNHj6pnz565e2Od3OLFi1W1alVVqVJFjz76qD7++GMZ/7/lyCOPPKJvvvlGiYmJZv9Vq1YpOTlZ3bp1kyRNmjRJCxYs0KxZs7R3714NHTpUjz76qH766Ser1xk5cqTeeOMN7du3T7Vq1VJiYqI6deqkNWvWaPv27erYsaO6dOmi6Oho85w+ffro9OnTWr9+vZYuXaoPP/xQ586ds7pujx49dO7cOX333XfaunWr6tWrp7Zt2youLi7b+12yZImuXLmS7Wdh8ODB8vHx0WeffXbTz+v8+fPl7e2t33//XW+99ZbGjx+v1atX2xTX4cOHtXTpUi1btkw7duzQmTNn1Lt3bw0YMED79u3T+vXr1b17d/PvA3kXHByslStXKiEhIdvnly1bpjJlymj8+PHm/xeSlJqaqvr162vFihXas2ePnnjiCT322GPavHmzJGnatGlq0qSJBg0aZJ5XtmzZLNe3x/+Lffr00Weffabp06dr3759+u9//ysfHx97vD3IB0WLFs22Mjhw4ECtXbtWx48fN9uOHj2qDRs2aODAgZKkYsWKad68efrzzz81bdo0ffTRR5oyZcq/iudmP7sBp+XoDOVW0bdvX8PV1dXw9va2Ojw9PW9YQWjUqJHx1FNPWV2rWbNmWSoI5cuXN65evWq29ejRw/xW4/jx44arq6tx6tQpq+u0bdvWGDVqlGEYhlGzZk1j7Nix2caeXWXgRv3+ujf9/zdr99133w3P++KLL4wSJUqYj3P6dvHv30L/8MMPhqurqxEdHW0+v3fvXkOSsXnz5hu+XmHQtGlTY+rUqYZhGMaVK1eMkiVLmt8q/fV4wYIFZv/evXubn4fU1FTDy8vL+PXXX62uOXDgQKN3796GYVz/u1y+fPlNY6levbrx3nvvGYZhGPv27TMkGVu2bDGfP3TokCHJ/Lv7+eefDV9fXyM1NdXqOhUrVjT++9//ZvsaQ4YMueE3zrVq1TLuueceq9j/+Xlt2bKl0bx5c6u2O++80xgxYkSu4xozZozh5uZmnDt3znx+69athiTj2LFjOcYHa7mtIPz0009GmTJlDDc3N6NBgwbG888/b2zcuNGqzz+rUznp3LmzMWzYMPNxdt/e/vOzk5f/F/9+3QMHDhiSslTZcGv4ewUhMzPTWL16teHh4WEMHz48S9+rV68aoaGhVlXHV155xShXrpyRkZGR7fXffvtto379+uZjWysIufnZDTgrKgh/07p1a+3YscPqmD179g3POXDggBo2bGjV9s/HklS9enW5urqaj0uXLm1+a7t7925lZGSocuXK8vHxMY+ffvpJR44ckSQ9++yzeu2119SsWTONGTNGu3btyvN9/vzzz9q6davmzZun/2vv7mOavL44gH9rW2pHHVgsKzW1FWs7iNXMmagz2DFrOhYSZ3whsxMVphvZcBqZJmK26BSSbTg2TOxicAPDlCXDmVATQZ0zEwMKiHEicdAGnNsQ40sqLrz07I+G5+dD63jzp7KdT0LC7du91NP79D7PPUez2QyXyyW6//jx41iwYAEmTpyIcePGYeXKlbh58yY6OzsH3UdjYyP0er3orF98fDwiIyPR2Ng47LGPBk1NTaipqcEbb7wBAJDJZEhJSUFhYaHQXr58OUpKSgAA9+7dw5EjR+B0OgEEzoB3dnZi4cKFongoLi4W4qHPrFmzRG2fz4esrCzExcUhMjISKpUKjY2NwhWEpqYmyGQyzJw5U3iOyWTC+PHjhXZDQwN8Ph+ioqJE/Xs8nqD+H7Xp06eL2g9+TgY7LoPBAI1GI7RnzJiBBQsWwGq1YtmyZdi3bx9u3br1f/07/m1aW1tF73lOTg4AYP78+WhpacGJEyewdOlS/PLLL0hISBgwCbi3txcff/wxrFYr1Go1VCoVjh07JrrSNRgjnRcvXLgAqVQKm802pOexx6e8vBwqlQpjx45FUlISUlJSkJycLIrHkpISSKVSrFq1Ct988w2ICH6/H0VFRVizZg3GjAl81SktLcW8efOg1WqhUqmwbdu2IcfcgwZz7GZstJI96QE8TcLDw2EymUS3Xbt27ZG8tlwuF7UlEgn8fj+AwJc6qVSK2tpa0SICgHCp+6233oLD4YDb7UZFRQVyc3ORl5eHzMzMIY9l8uTJiIyMhMViQXt7O1JSUnD69GkAgNfrRXJyMjIyMrBr1y6o1Wr8/PPPSE9PR1dXFychD0JhYSF6enqg0+mE24gICoUCe/bsQUREBJxOJ2w2G9rb21FZWQmlUolXX30VAIStR263GxMnThS9tkKhELXDw8NF7aysLFRWVuKzzz6DyWSCUqnE0qVLh5Ss6/P5EBMTg1OnTgXd97ASv2azGXfu3MH169dFfzcQSBRubm5GYmLigH0P9DkZzLj6vydSqRSVlZWoqqpCRUUFCgoKkJ2djerqakyePHnAMTFAp9PhwoULQlutVgu/y+VyJCQkICEhAVu2bMHOnTuxY8cObNmyRbR18UGffvopvvjiC+Tn58NqtSI8PBwbNmwYclL5SOdFpVI5pP7Y45eYmIi9e/ciLCwMOp0OMpkM9+/fF8Xjc889BwBIS0tDbm4uTp48Cb/fj7a2NqxZswZAoKqc0+nE9u3b4XA4EBERgUOHDiEvL++hfY8ZMyZoK2J3d7fw+2CO3YyNVnwFYYQsFgvOnTsnuq1/eyAvvPACent70d7eDpPJJPrRarXC4/R6vVA1ZNOmTdi3bx+A/+UP9Pb2Dnn87777Li5duoTDhw8DAGpra+H3+5GXl4c5c+bAbDYHVeAJCwsbsK+4uDi0tbWhra1NuO3y5cu4ffs24uPjhzzO0aKnpwfFxcXIy8sTXYlqaGiATqfDwYMHAQRyOfR6PUpLS1FSUoJly5YJX47j4+OhUCjQ2toaFA+h9mE/6MyZM1i9ejUWL14Mq9UKrVYLr9cr3G+xWNDT04P6+nrhtl9//VV0Rn3mzJn4448/IJPJgvqfMGFCyH6XLFkCuVwe8mDrcrlw79494YrKcON1OOPqI5FIMG/ePGzfvh319fUICwsTYp4NrP97/uACob/4+Hj09PQIJSdDzRdnzpzBokWL8Oabb2LGjBmIjY0NKh89mHkGGNm8aLVa4ff7g3J72NOj78TdpEmTIJMFzmkqlUpRPPZVpZoyZQpsNhv279+Pr7/+Gna7HQaDAQBQVVUFg8GA7OxszJo1C1OnThXlK4Si0WiEvBkgEEuXLl0S2oM9djM2GvECYYQyMzNRWFiIoqIiXL16FTt37sTFixeHVIbPbDbD6XQiNTUVZWVl8Hg8qKmpQW5uLtxuNwBgw4YNOHbsGDweD+rq6vDjjz8iLi4OQGBLhUQiQXl5OW7cuCFKfh3IM888g7Vr1+Kjjz4CEcFkMqG7uxsFBQVoaWnBgQMHgrYgGY1G+Hw+nDhxAh0dHSG3HtntdlitVjidTtTV1aGmpgapqamw2WxB22L+TcrLy3Hr1i2kp6dj2rRpop8lS5YI24wAYMWKFXC5XKisrBS2FwGBRLqsrCxs3LgRRUVFaG5uRl1dHQoKClBUVPSP/U+dOlVIzm1oaMCKFSuEM/AA8Pzzz8Nut2PdunWoqalBfX091q1bB6VSKcSs3W7H3Llz8frrr6OiogJerxdVVVXIzs7G+fPnQ/Y7adIkfPLJJ8jPz0d2djauXLmC5uZm7N69G5s3b8amTZswe/ZsAMOP1+GMCwCqq6uRk5OD8+fPo7W1FWVlZbhx44bw+WHD9/LLL+Orr75CbW0tvF4vjh49iq1btyIxMRHPPvssgMB8cfr0afz222/o6OgAEIjTvqs6jY2NePvtt/Hnn3+KXttoNKK6uhperxcdHR2iOO4z0nnRaDRi1apVSEtLww8//ACPx4NTp07hu+++e9RvFXtM0tPTUVZWhsOHDwvJyUAg5lpbW3Ho0CE0Nzfjyy+/HPAkwSuvvAK32w23240rV64gIyND9P9qDObYzdio9WRTIJ4eIylzumPHDpowYQKpVCpKS0uj9evXi8qwhXrt999/n2w2m9Du6uqiDz/8kIxGI8nlcoqJiaHFixfTxYsXiYjovffeoylTppBCoSCNRkMrV66kjo4O0Ri0Wi1JJJIhlTklCpSolMlkVFpaSkREu3fvppiYGFIqleRwOKi4uDjoee+88w5FRUVxmdN+kpOT6bXXXgt5X3V1taik7OXLlwkAGQwG8vv9osf6/X7Kz88ni8VCcrmcNBoNORwO+umnn4jo4f+WHo+HEhMTSalUkl6vpz179gQl2l2/fp2SkpJIoVCQwWCgb7/9lqKjo8nlcgmPuXv3LmVmZpJOpyO5XE56vZ6cTqco6TyUI0eOUEJCgpDg/+KLL9L+/fuDHhcqXgdTUnCgcfVPMux7nx0OB2k0GlIoFGQ2m4WkbRbaYJOUc3JyaO7cuaRWq2ns2LEUGxtL69evF81NZ8+epenTp5NCoRDKnN68eZMWLVpEKpWKoqOjadu2bZSamiqaJ5uammjOnDmkVCofWuZ0OPNi/zi7f/8+bdy4kWJiYigsLIxMJlPImGWP38OOy/+ks7OTIiIiSK1WBxU0+OCDDygqKopUKhWlpKTQ559/Lorz/vNHV1cXZWRkkFqtpujoaMrNzQ2akwY6djM2WkmIuNbfo7Zw4UJotVocOHDgSQ+FsQFdu3YNer1eSE5njDHG2H8bJymPUGdnJ1wuFxwOB6RSKQ4ePIjjx4+Larcz9jQ5efIkfD4frFYrfv/9d2zevBlGoxHz589/0kNjjDHG2FOAFwgjJJFIcPToUezatQt//fUXLBYLvv/+e9jt9ic9NMZC6u7uxtatW9HS0oJx48bhpZdeQklJSVAFIcYYY4z9N/EWI8YYY4wxxpiAqxgxxhhjjDHGBLxAYIwxxhhjjAl4gcAYY4wxxhgT8AKBMcYYY4wxJuAFAmOMMcYYY0zACwTGGGOMMcaYgBcIjDHGGGOMMQEvEBhjjDHGGGMCXiAwxhhjjDHGBH8D6S4jx6WZxOgAAAAASUVORK5CYII=", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "T-test for Highest Ratio: T-Statistic = -57.59965843801415, P-Value = 0.0\n", + "T-test for Average Others: T-Statistic = -21.080776226637518, P-Value = 1.2478046488137352e-93\n", + "T-test for T-Statistic: T-Statistic = nan, P-Value = nan\n", + "T-test for P-Value: T-Statistic = nan, P-Value = nan\n" + ] + }, + { + "ename": "ValueError", + "evalue": "Input X contains NaN.\nRandomForestClassifier does not accept missing values encoded as NaN natively. For supervised learning, you might want to consider sklearn.ensemble.HistGradientBoostingClassifier and Regressor which accept missing values encoded as NaNs natively. Alternatively, it is possible to preprocess the data, for instance by using an imputer transformer in a pipeline or drop samples with missing values. See https://scikit-learn.org/stable/modules/impute.html You can find a list of all estimators that handle NaN values at the following page: https://scikit-learn.org/stable/modules/impute.html#estimators-that-handle-nan-values", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mValueError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[1;32mIn[46], line 91\u001b[0m\n\u001b[0;32m 89\u001b[0m \u001b[38;5;66;03m# Train a RandomForestClassifier\u001b[39;00m\n\u001b[0;32m 90\u001b[0m clf \u001b[38;5;241m=\u001b[39m RandomForestClassifier(random_state\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m42\u001b[39m)\n\u001b[1;32m---> 91\u001b[0m \u001b[43mclf\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfit\u001b[49m\u001b[43m(\u001b[49m\u001b[43mX_train\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43my_train\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 93\u001b[0m \u001b[38;5;66;03m# Make predictions\u001b[39;00m\n\u001b[0;32m 94\u001b[0m y_pred \u001b[38;5;241m=\u001b[39m clf\u001b[38;5;241m.\u001b[39mpredict(X_test)\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\sklearn\\base.py:1152\u001b[0m, in \u001b[0;36m_fit_context..decorator..wrapper\u001b[1;34m(estimator, *args, **kwargs)\u001b[0m\n\u001b[0;32m 1145\u001b[0m estimator\u001b[38;5;241m.\u001b[39m_validate_params()\n\u001b[0;32m 1147\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m config_context(\n\u001b[0;32m 1148\u001b[0m skip_parameter_validation\u001b[38;5;241m=\u001b[39m(\n\u001b[0;32m 1149\u001b[0m prefer_skip_nested_validation \u001b[38;5;129;01mor\u001b[39;00m global_skip_validation\n\u001b[0;32m 1150\u001b[0m )\n\u001b[0;32m 1151\u001b[0m ):\n\u001b[1;32m-> 1152\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m fit_method(estimator, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\sklearn\\ensemble\\_forest.py:348\u001b[0m, in \u001b[0;36mBaseForest.fit\u001b[1;34m(self, X, y, sample_weight)\u001b[0m\n\u001b[0;32m 346\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m issparse(y):\n\u001b[0;32m 347\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124msparse multilabel-indicator for y is not supported.\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m--> 348\u001b[0m X, y \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_validate_data\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 349\u001b[0m \u001b[43m \u001b[49m\u001b[43mX\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43my\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mmulti_output\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mTrue\u001b[39;49;00m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43maccept_sparse\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mcsc\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdtype\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mDTYPE\u001b[49m\n\u001b[0;32m 350\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 351\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m sample_weight \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[0;32m 352\u001b[0m sample_weight \u001b[38;5;241m=\u001b[39m _check_sample_weight(sample_weight, X)\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\sklearn\\base.py:622\u001b[0m, in \u001b[0;36mBaseEstimator._validate_data\u001b[1;34m(self, X, y, reset, validate_separately, cast_to_ndarray, **check_params)\u001b[0m\n\u001b[0;32m 620\u001b[0m y \u001b[38;5;241m=\u001b[39m check_array(y, input_name\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124my\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mcheck_y_params)\n\u001b[0;32m 621\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m--> 622\u001b[0m X, y \u001b[38;5;241m=\u001b[39m check_X_y(X, y, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mcheck_params)\n\u001b[0;32m 623\u001b[0m out \u001b[38;5;241m=\u001b[39m X, y\n\u001b[0;32m 625\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m no_val_X \u001b[38;5;129;01mand\u001b[39;00m check_params\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mensure_2d\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;28;01mTrue\u001b[39;00m):\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\sklearn\\utils\\validation.py:1146\u001b[0m, in \u001b[0;36mcheck_X_y\u001b[1;34m(X, y, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, multi_output, ensure_min_samples, ensure_min_features, y_numeric, estimator)\u001b[0m\n\u001b[0;32m 1141\u001b[0m estimator_name \u001b[38;5;241m=\u001b[39m _check_estimator_name(estimator)\n\u001b[0;32m 1142\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[0;32m 1143\u001b[0m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mestimator_name\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m requires y to be passed, but the target y is None\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m 1144\u001b[0m )\n\u001b[1;32m-> 1146\u001b[0m X \u001b[38;5;241m=\u001b[39m \u001b[43mcheck_array\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 1147\u001b[0m \u001b[43m \u001b[49m\u001b[43mX\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1148\u001b[0m \u001b[43m \u001b[49m\u001b[43maccept_sparse\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43maccept_sparse\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1149\u001b[0m \u001b[43m \u001b[49m\u001b[43maccept_large_sparse\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43maccept_large_sparse\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1150\u001b[0m \u001b[43m \u001b[49m\u001b[43mdtype\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdtype\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1151\u001b[0m \u001b[43m \u001b[49m\u001b[43morder\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43morder\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1152\u001b[0m \u001b[43m \u001b[49m\u001b[43mcopy\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcopy\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1153\u001b[0m \u001b[43m \u001b[49m\u001b[43mforce_all_finite\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mforce_all_finite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1154\u001b[0m \u001b[43m \u001b[49m\u001b[43mensure_2d\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mensure_2d\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1155\u001b[0m \u001b[43m \u001b[49m\u001b[43mallow_nd\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mallow_nd\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1156\u001b[0m \u001b[43m \u001b[49m\u001b[43mensure_min_samples\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mensure_min_samples\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1157\u001b[0m \u001b[43m \u001b[49m\u001b[43mensure_min_features\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mensure_min_features\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1158\u001b[0m \u001b[43m \u001b[49m\u001b[43mestimator\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mestimator\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1159\u001b[0m \u001b[43m \u001b[49m\u001b[43minput_name\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mX\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1160\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 1162\u001b[0m y \u001b[38;5;241m=\u001b[39m _check_y(y, multi_output\u001b[38;5;241m=\u001b[39mmulti_output, y_numeric\u001b[38;5;241m=\u001b[39my_numeric, estimator\u001b[38;5;241m=\u001b[39mestimator)\n\u001b[0;32m 1164\u001b[0m check_consistent_length(X, y)\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\sklearn\\utils\\validation.py:957\u001b[0m, in \u001b[0;36mcheck_array\u001b[1;34m(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, estimator, input_name)\u001b[0m\n\u001b[0;32m 951\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[0;32m 952\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mFound array with dim \u001b[39m\u001b[38;5;132;01m%d\u001b[39;00m\u001b[38;5;124m. \u001b[39m\u001b[38;5;132;01m%s\u001b[39;00m\u001b[38;5;124m expected <= 2.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m 953\u001b[0m \u001b[38;5;241m%\u001b[39m (array\u001b[38;5;241m.\u001b[39mndim, estimator_name)\n\u001b[0;32m 954\u001b[0m )\n\u001b[0;32m 956\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m force_all_finite:\n\u001b[1;32m--> 957\u001b[0m \u001b[43m_assert_all_finite\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 958\u001b[0m \u001b[43m \u001b[49m\u001b[43marray\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 959\u001b[0m \u001b[43m \u001b[49m\u001b[43minput_name\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43minput_name\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 960\u001b[0m \u001b[43m \u001b[49m\u001b[43mestimator_name\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mestimator_name\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 961\u001b[0m \u001b[43m \u001b[49m\u001b[43mallow_nan\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mforce_all_finite\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m==\u001b[39;49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mallow-nan\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[0;32m 962\u001b[0m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 964\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m ensure_min_samples \u001b[38;5;241m>\u001b[39m \u001b[38;5;241m0\u001b[39m:\n\u001b[0;32m 965\u001b[0m n_samples \u001b[38;5;241m=\u001b[39m _num_samples(array)\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\sklearn\\utils\\validation.py:122\u001b[0m, in \u001b[0;36m_assert_all_finite\u001b[1;34m(X, allow_nan, msg_dtype, estimator_name, input_name)\u001b[0m\n\u001b[0;32m 119\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m first_pass_isfinite:\n\u001b[0;32m 120\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m\n\u001b[1;32m--> 122\u001b[0m \u001b[43m_assert_all_finite_element_wise\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m 123\u001b[0m \u001b[43m \u001b[49m\u001b[43mX\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 124\u001b[0m \u001b[43m \u001b[49m\u001b[43mxp\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mxp\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 125\u001b[0m \u001b[43m \u001b[49m\u001b[43mallow_nan\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mallow_nan\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 126\u001b[0m \u001b[43m \u001b[49m\u001b[43mmsg_dtype\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmsg_dtype\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 127\u001b[0m \u001b[43m \u001b[49m\u001b[43mestimator_name\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mestimator_name\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 128\u001b[0m \u001b[43m \u001b[49m\u001b[43minput_name\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43minput_name\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 129\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n", + "File \u001b[1;32m~\\.conda\\envs\\py310\\lib\\site-packages\\sklearn\\utils\\validation.py:171\u001b[0m, in \u001b[0;36m_assert_all_finite_element_wise\u001b[1;34m(X, xp, allow_nan, msg_dtype, estimator_name, input_name)\u001b[0m\n\u001b[0;32m 154\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m estimator_name \u001b[38;5;129;01mand\u001b[39;00m input_name \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mX\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;129;01mand\u001b[39;00m has_nan_error:\n\u001b[0;32m 155\u001b[0m \u001b[38;5;66;03m# Improve the error message on how to handle missing values in\u001b[39;00m\n\u001b[0;32m 156\u001b[0m \u001b[38;5;66;03m# scikit-learn.\u001b[39;00m\n\u001b[0;32m 157\u001b[0m msg_err \u001b[38;5;241m+\u001b[39m\u001b[38;5;241m=\u001b[39m (\n\u001b[0;32m 158\u001b[0m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;132;01m{\u001b[39;00mestimator_name\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m does not accept missing values\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m 159\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m encoded as NaN natively. For supervised learning, you might want\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 169\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m#estimators-that-handle-nan-values\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m 170\u001b[0m )\n\u001b[1;32m--> 171\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(msg_err)\n", + "\u001b[1;31mValueError\u001b[0m: Input X contains NaN.\nRandomForestClassifier does not accept missing values encoded as NaN natively. For supervised learning, you might want to consider sklearn.ensemble.HistGradientBoostingClassifier and Regressor which accept missing values encoded as NaNs natively. Alternatively, it is possible to preprocess the data, for instance by using an imputer transformer in a pipeline or drop samples with missing values. See https://scikit-learn.org/stable/modules/impute.html You can find a list of all estimators that handle NaN values at the following page: https://scikit-learn.org/stable/modules/impute.html#estimators-that-handle-nan-values" + ] + } + ], + "source": [ + "import pandas as pd\n", + "import seaborn as sns\n", + "import matplotlib.pyplot as plt\n", + "from scipy.stats import ttest_ind\n", + "from sklearn.model_selection import train_test_split\n", + "from sklearn.ensemble import RandomForestClassifier\n", + "from sklearn.metrics import classification_report, confusion_matrix\n", + "\n", + "# Assuming list_of_significance and list_of_significance_watermarked are already defined\n", + "# Create DataFrames from the lists\n", + "df_significance = pd.DataFrame(list_of_significance, columns=['Highest Ratio', 'Average Others', 'T-Statistic', 'P-Value'])\n", + "df_significance_watermarked = pd.DataFrame(list_of_significance_watermarked, columns=['Highest Ratio', 'Average Others', 'T-Statistic', 'P-Value'])\n", + "\n", + "# Add a label column to distinguish between the two sets\n", + "df_significance['Label'] = 'Original'\n", + "df_significance_watermarked['Label'] = 'Watermarked'\n", + "\n", + "# Combine the DataFrames\n", + "combined_df = pd.concat([df_significance, df_significance_watermarked], ignore_index=True)\n", + "\n", + "# Perform EDA\n", + "def perform_eda(df):\n", + " # Display the first few rows of the DataFrame\n", + " print(\"First few rows of the DataFrame:\")\n", + " print(df.head())\n", + "\n", + " # Display statistical summary\n", + " print(\"\\nStatistical Summary:\")\n", + " print(df.describe())\n", + "\n", + " # Check for missing values\n", + " print(\"\\nMissing Values:\")\n", + " print(df.isnull().sum())\n", + "\n", + " # Visualize the distributions of the features\n", + " plt.figure(figsize=(12, 8))\n", + " sns.histplot(data=df, x='Highest Ratio', hue='Label', element='step', kde=True)\n", + " plt.title('Distribution of Highest Ratio')\n", + " plt.show()\n", + "\n", + " plt.figure(figsize=(12, 8))\n", + " sns.histplot(data=df, x='Average Others', hue='Label', element='step', kde=True)\n", + " plt.title('Distribution of Average Others')\n", + " plt.show()\n", + "\n", + " plt.figure(figsize=(12, 8))\n", + " sns.histplot(data=df, x='T-Statistic', hue='Label', element='step', kde=True)\n", + " plt.title('Distribution of T-Statistic')\n", + " plt.show()\n", + "\n", + " plt.figure(figsize=(12, 8))\n", + " sns.histplot(data=df, x='P-Value', hue='Label', element='step', kde=True)\n", + " plt.title('Distribution of P-Value')\n", + " plt.show()\n", + "\n", + " # Pairplot to see relationships\n", + " sns.pairplot(df, hue='Label')\n", + " plt.show()\n", + "\n", + " # Correlation matrix\n", + " plt.figure(figsize=(10, 8))\n", + " sns.heatmap(df.drop(columns=['Label']).corr(), annot=True, cmap='coolwarm')\n", + " plt.title('Correlation Matrix')\n", + " plt.show()\n", + "\n", + " # T-test to check for significant differences\n", + " original = df[df['Label'] == 'Original']\n", + " watermarked = df[df['Label'] == 'Watermarked']\n", + "\n", + " for column in ['Highest Ratio', 'Average Others', 'T-Statistic', 'P-Value']:\n", + " t_stat, p_value = ttest_ind(original[column], watermarked[column])\n", + " print(f\"T-test for {column}: T-Statistic = {t_stat}, P-Value = {p_value}\")\n", + "\n", + "# Perform EDA on the combined DataFrame\n", + "perform_eda(combined_df)\n", + "\n", + "# Check if the data is ready for machine learning classification\n", + "\n", + "# Prepare the data\n", + "X = combined_df.drop(columns=['Label'])\n", + "y = combined_df['Label']\n", + "\n", + "# Convert labels to numerical values for ML model\n", + "y = y.map({'Original': 0, 'Watermarked': 1})\n", + "\n", + "# Split the data into training and testing sets\n", + "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n", + "\n", + "# Train a RandomForestClassifier\n", + "clf = RandomForestClassifier(random_state=42)\n", + "clf.fit(X_train, y_train)\n", + "\n", + "# Make predictions\n", + "y_pred = clf.predict(X_test)\n", + "\n", + "# Evaluate the model\n", + "print(\"\\nClassification Report:\")\n", + "print(classification_report(y_test, y_pred))\n", + "\n", + "print(\"\\nConfusion Matrix:\")\n", + "print(confusion_matrix(y_test, y_pred))\n", + "\n", + "# Feature importances\n", + "feature_importances = clf.feature_importances_\n", + "\n", + "# Create a DataFrame for feature importances\n", + "feature_importances_df = pd.DataFrame({\n", + " 'Feature': X.columns,\n", + " 'Importance': feature_importances\n", + "}).sort_values(by='Importance', ascending=False)\n", + "\n", + "# Plot feature importances\n", + "plt.figure(figsize=(12, 8))\n", + "sns.barplot(x='Importance', y='Feature', data=feature_importances_df, palette='viridis')\n", + "plt.title('Feature Importances')\n", + "plt.show()\n", + "\n", + "# Heatmap for feature importances\n", + "plt.figure(figsize=(10, 8))\n", + "sns.heatmap(feature_importances_df.set_index('Feature').T, annot=True, cmap='viridis')\n", + "plt.title('Heatmap of Feature Importances')\n", + "plt.show()\n" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "DN8bw7WqUXX5", + "outputId": "10c5dd1f-0e8d-43a7-82e0-02dea500dbe8" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "Logistic Regression Classification Report:\n", + " precision recall f1-score support\n", + "\n", + " 0 0.83 0.87 0.85 415\n", + " 1 0.85 0.81 0.83 385\n", + "\n", + " accuracy 0.84 800\n", + " macro avg 0.84 0.84 0.84 800\n", + "weighted avg 0.84 0.84 0.84 800\n", + "\n", + "\n", + "Logistic Regression Confusion Matrix:\n", + "[[360 55]\n", + " [ 73 312]]\n", + "\n", + "Decision Tree Classification Report:\n", + " precision recall f1-score support\n", + "\n", + " 0 0.92 0.92 0.92 415\n", + " 1 0.91 0.91 0.91 385\n", + "\n", + " accuracy 0.91 800\n", + " macro avg 0.91 0.91 0.91 800\n", + "weighted avg 0.91 0.91 0.91 800\n", + "\n", + "\n", + "Decision Tree Confusion Matrix:\n", + "[[380 35]\n", + " [ 35 350]]\n", + "\n", + "Random Forest Classification Report:\n", + " precision recall f1-score support\n", + "\n", + " 0 0.91 0.94 0.93 415\n", + " 1 0.94 0.90 0.92 385\n", + "\n", + " accuracy 0.92 800\n", + " macro avg 0.92 0.92 0.92 800\n", + "weighted avg 0.92 0.92 0.92 800\n", + "\n", + "\n", + "Random Forest Confusion Matrix:\n", + "[[391 24]\n", + " [ 39 346]]\n", + "\n", + "Support Vector Machine Classification Report:\n", + " precision recall f1-score support\n", + "\n", + " 0 0.71 0.79 0.75 415\n", + " 1 0.74 0.66 0.70 385\n", + "\n", + " accuracy 0.73 800\n", + " macro avg 0.73 0.72 0.72 800\n", + "weighted avg 0.73 0.72 0.72 800\n", + "\n", + "\n", + "Support Vector Machine Confusion Matrix:\n", + "[[327 88]\n", + " [132 253]]\n", + "\n", + "Gradient Boosting Classification Report:\n", + " precision recall f1-score support\n", + "\n", + " 0 0.93 0.94 0.94 415\n", + " 1 0.94 0.92 0.93 385\n", + "\n", + " accuracy 0.94 800\n", + " macro avg 0.94 0.93 0.93 800\n", + "weighted avg 0.94 0.94 0.93 800\n", + "\n", + "\n", + "Gradient Boosting Confusion Matrix:\n", + "[[392 23]\n", + " [ 29 356]]\n", + "\n", + "AdaBoost Classification Report:\n", + " precision recall f1-score support\n", + "\n", + " 0 0.90 0.91 0.90 415\n", + " 1 0.90 0.89 0.89 385\n", + "\n", + " accuracy 0.90 800\n", + " macro avg 0.90 0.90 0.90 800\n", + "weighted avg 0.90 0.90 0.90 800\n", + "\n", + "\n", + "AdaBoost Confusion Matrix:\n", + "[[376 39]\n", + " [ 44 341]]\n", + "\n", + "Naive Bayes Classification Report:\n", + " precision recall f1-score support\n", + "\n", + " 0 0.78 0.81 0.79 415\n", + " 1 0.78 0.76 0.77 385\n", + "\n", + " accuracy 0.78 800\n", + " macro avg 0.78 0.78 0.78 800\n", + "weighted avg 0.78 0.78 0.78 800\n", + "\n", + "\n", + "Naive Bayes Confusion Matrix:\n", + "[[335 80]\n", + " [ 94 291]]\n", + "\n", + "K-Nearest Neighbors Classification Report:\n", + " precision recall f1-score support\n", + "\n", + " 0 0.82 0.87 0.84 415\n", + " 1 0.85 0.79 0.82 385\n", + "\n", + " accuracy 0.83 800\n", + " macro avg 0.83 0.83 0.83 800\n", + "weighted avg 0.83 0.83 0.83 800\n", + "\n", + "\n", + "K-Nearest Neighbors Confusion Matrix:\n", + "[[361 54]\n", + " [ 81 304]]\n" + ] + } + ], + "source": [ + "import pandas as pd\n", + "import seaborn as sns\n", + "import matplotlib.pyplot as plt\n", + "from sklearn.model_selection import train_test_split\n", + "from sklearn.linear_model import LogisticRegression\n", + "from sklearn.tree import DecisionTreeClassifier\n", + "from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier, AdaBoostClassifier\n", + "from sklearn.svm import SVC\n", + "from sklearn.naive_bayes import GaussianNB\n", + "from sklearn.neighbors import KNeighborsClassifier\n", + "from sklearn.metrics import classification_report, confusion_matrix\n", + "\n", + "# Assuming list_of_significance and list_of_significance_watermarked are already defined\n", + "# Create DataFrames from the lists\n", + "df_significance = pd.DataFrame(list_of_significance, columns=['Highest Ratio', 'Average Others', 'T-Statistic', 'P-Value'])\n", + "df_significance_watermarked = pd.DataFrame(list_of_significance_watermarked, columns=['Highest Ratio', 'Average Others', 'T-Statistic', 'P-Value'])\n", + "\n", + "# Add a label column to distinguish between the two sets\n", + "df_significance['Label'] = 'Original'\n", + "df_significance_watermarked['Label'] = 'Watermarked'\n", + "\n", + "# Combine the DataFrames\n", + "combined_df = pd.concat([df_significance, df_significance_watermarked], ignore_index=True)\n", + "combined_df = combined_df.dropna()\n", + "\n", + "# Prepare the data\n", + "X = combined_df.drop(columns=['Label'])\n", + "y = combined_df['Label']\n", + "\n", + "# Convert labels to numerical values for ML model\n", + "y = y.map({'Original': 0, 'Watermarked': 1})\n", + "\n", + "# Split the data into training and testing sets\n", + "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n", + "\n", + "# Initialize models\n", + "models = {\n", + " 'Logistic Regression': LogisticRegression(random_state=42, max_iter=1000),\n", + " 'Decision Tree': DecisionTreeClassifier(random_state=42),\n", + " 'Random Forest': RandomForestClassifier(random_state=42),\n", + " 'Support Vector Machine': SVC(random_state=42),\n", + " 'Gradient Boosting': GradientBoostingClassifier(random_state=42),\n", + " 'AdaBoost': AdaBoostClassifier(random_state=42),\n", + " 'Naive Bayes': GaussianNB(),\n", + " 'K-Nearest Neighbors': KNeighborsClassifier()\n", + "}\n", + "\n", + "# Train and evaluate models\n", + "for model_name, model in models.items():\n", + " model.fit(X_train, y_train)\n", + " y_pred = model.predict(X_test)\n", + " print(f\"\\n{model_name} Classification Report:\")\n", + " print(classification_report(y_test, y_pred))\n", + " print(f\"\\n{model_name} Confusion Matrix:\")\n", + " print(confusion_matrix(y_test, y_pred))\n", + "\n", + " # Feature importances (only for models that provide it)\n", + " if hasattr(model, 'feature_importances_'):\n", + " feature_importances = model.feature_importances_\n", + " feature_importances_df = pd.DataFrame({\n", + " 'Feature': X.columns,\n", + " 'Importance': feature_importances\n", + " }).sort_values(by='Importance', ascending=False)\n", + "\n", + " # Plot feature importances\n", + " # plt.figure(figsize=(12, 8))\n", + " # sns.barplot(x='Importance', y='Feature', data=feature_importances_df, palette='viridis')\n", + " # plt.title(f'{model_name} Feature Importances')\n", + " plt.show()\n" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": { + "id": "sJxGEZAJzPmz" + }, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "hOnM5F1LXklH", + "outputId": "97eb0191-4324-40a5-e7f6-4479ad4a3443" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Number of test cases created: 2000\n" + ] + } + ], + "source": [ + "import os\n", + "import random\n", + "\n", + "def extract_test_cases(folder_path, num_cases=2000, words_per_case=300):\n", + " test_cases = []\n", + " book_files = [f for f in os.listdir(folder_path) if os.path.isfile(os.path.join(folder_path, f))]\n", + "\n", + " # Calculate the number of test cases to extract from each book\n", + " cases_per_book = num_cases // len(book_files)\n", + " extra_cases = num_cases % len(book_files)\n", + "\n", + " for book_file in book_files:\n", + " with open(os.path.join(folder_path, book_file), 'r', encoding='utf-8') as file:\n", + " text = file.read()\n", + " words = text.split()\n", + " num_words = len(words)\n", + "\n", + " # Ensure enough words are available to extract the cases\n", + " if num_words < words_per_case:\n", + " continue\n", + "\n", + " # Determine the number of cases to extract from this book\n", + " num_cases_from_book = cases_per_book\n", + " if extra_cases > 0:\n", + " num_cases_from_book += 1\n", + " extra_cases -= 1\n", + "\n", + " for _ in range(num_cases_from_book):\n", + " start_index = random.randint(0, num_words - words_per_case)\n", + " case = ' '.join(words[start_index:start_index + words_per_case])\n", + " test_cases.append(case)\n", + "\n", + " if len(test_cases) == num_cases:\n", + " return test_cases\n", + "\n", + " return test_cases\n", + "\n", + "# Usage example\n", + "folder_path = 'books'\n", + "test_cases = extract_test_cases(folder_path)\n", + "\n", + "# Output the number of test cases created\n", + "print(f\"Number of test cases created: {len(test_cases)}\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": { + "id": "9NEvIc_HY43Z" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "___________________________________________________________________________________________________________________________\n", + "Doing 1\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.22814207650273222\n", + "T-Statistic: -21.334991021776784\n", + "P-Value: 0.00022530414214046572\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.1822033898305085\n", + "T-Statistic: -3.53275826407369\n", + "P-Value: 0.038562976693981454\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 2\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4098360655737705\n", + "Average of Other Ratios: 0.23333333333333334\n", + "T-Statistic: -4.992251154606664\n", + "P-Value: 0.015458009685690827\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.1391949152542373\n", + "T-Statistic: -3.4405910948750495\n", + "P-Value: 0.04121820653114378\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 2\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 3\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.34815573770491803\n", + "T-Statistic: -6.977885499593617\n", + "P-Value: 0.0060406875581721555\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.27033898305084747\n", + "T-Statistic: -2.228607614649941\n", + "P-Value: 0.11214158967770235\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 3\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 4\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.2573087431693989\n", + "T-Statistic: -17.794177111160675\n", + "P-Value: 0.0003870090924213516\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.16836158192090395\n", + "T-Statistic: -2.451612903225806\n", + "P-Value: 0.2465587655124727\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 4\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 5\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3151639344262295\n", + "T-Statistic: -1.713189822924711\n", + "P-Value: 0.18519433572899746\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.21087570621468926\n", + "T-Statistic: -5.467540160267347\n", + "P-Value: 0.012025943288987453\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 5\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 6\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.20416666666666666\n", + "T-Statistic: -8.101361023294555\n", + "P-Value: 0.003930735409185079\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.1307909604519774\n", + "T-Statistic: -11.145126479863883\n", + "P-Value: 0.0015479966208348658\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 6\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 7\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.27424863387978143\n", + "T-Statistic: -2.647512144273123\n", + "P-Value: 0.07715790266759627\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.1603813559322034\n", + "T-Statistic: -4.047402698396378\n", + "P-Value: 0.027156785257683596\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 7\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 8\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.28599726775956286\n", + "T-Statistic: -9.817142706536112\n", + "P-Value: 0.002246603044354501\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3\n", + "Average of Other Ratios: 0.22033898305084745\n", + "T-Statistic: -11.51260179108094\n", + "P-Value: 0.0014069485474090153\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 8\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 9\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.2375\n", + "T-Statistic: -5.570367388129549\n", + "P-Value: 0.011418116056075428\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.18149717514124292\n", + "T-Statistic: -3.7964977175244834\n", + "P-Value: 0.03208088709594881\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 9\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 10\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.2775273224043716\n", + "T-Statistic: -4.727075685541707\n", + "P-Value: 0.01793917650756737\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.16871468926553673\n", + "T-Statistic: -3.7922455055393622\n", + "P-Value: 0.03217383630567124\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 10\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 11\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4098360655737705\n", + "Average of Other Ratios: 0.25\n", + "T-Statistic: -3.714254520543179\n", + "P-Value: 0.033941551397426564\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.1398305084745763\n", + "T-Statistic: -4.975896705378727\n", + "P-Value: 0.015597602000975219\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 11\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 12\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.24904371584699453\n", + "T-Statistic: -14.463091326070671\n", + "P-Value: 0.0007165776065197027\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.20261299435028246\n", + "T-Statistic: -3.3817063110386885\n", + "P-Value: 0.04303714816975945\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 12\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 13\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.275\n", + "T-Statistic: -5.83498532451519\n", + "P-Value: 0.01002850287511932\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.18177966101694915\n", + "T-Statistic: -3.365869501933496\n", + "P-Value: 0.04354367094755919\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 13\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 14\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.23750000000000002\n", + "T-Statistic: -15.536893060799459\n", + "P-Value: 0.0005793474370025991\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.1477401129943503\n", + "T-Statistic: -3.4073375272246085\n", + "P-Value: 0.04223311481214282\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 14\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 15\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.32745901639344266\n", + "T-Statistic: -6.77625507348341\n", + "P-Value: 0.006568385846444286\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.27838983050847455\n", + "T-Statistic: -2.9307183932115923\n", + "P-Value: 0.06096526759447833\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 15\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 16\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.21666666666666667\n", + "T-Statistic: -4.119026835630454\n", + "P-Value: 0.025932160329463834\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.16666666666666666\n", + "Average of Other Ratios: 0.1228813559322034\n", + "T-Statistic: -10.333333333333332\n", + "P-Value: 0.001933293191806968\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 16\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 17\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.23333333333333334\n", + "T-Statistic: -9.613578441019637\n", + "P-Value: 0.0023886490069146135\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.21101694915254238\n", + "T-Statistic: -4.384236405710172\n", + "P-Value: 0.02197310950253267\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 17\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 18\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.2775956284153005\n", + "T-Statistic: -2.196141651943659\n", + "P-Value: 0.11558815206376069\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.14745762711864407\n", + "T-Statistic: -2.9054879908745583\n", + "P-Value: 0.062224127599699926\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 18\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 19\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.29480874316939887\n", + "T-Statistic: -3.9127157656292244\n", + "P-Value: 0.029668656491470317\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.21087570621468926\n", + "T-Statistic: -2.5909821905688375\n", + "P-Value: 0.08100515899541934\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 19\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 20\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3770491803278688\n", + "Average of Other Ratios: 0.2125\n", + "T-Statistic: -4.70897478231848\n", + "P-Value: 0.018126865049367218\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.21666666666666667\n", + "Average of Other Ratios: 0.1440677966101695\n", + "T-Statistic: -2.967580383634676\n", + "P-Value: 0.05918282683371976\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 20\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 21\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3523907103825137\n", + "T-Statistic: -5.470026246143757\n", + "P-Value: 0.012010754748633298\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3206920903954802\n", + "T-Statistic: -4.689814564762172\n", + "P-Value: 0.018328326818943686\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 21\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 22\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3442622950819672\n", + "Average of Other Ratios: 0.2875\n", + "T-Statistic: -2.0013563154719005\n", + "P-Value: 0.13914298161809877\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.13940677966101694\n", + "T-Statistic: -5.09630233956434\n", + "P-Value: 0.014606958299961344\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 22\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 23\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.22786885245901642\n", + "T-Statistic: -2.9204148617045544\n", + "P-Value: 0.06147547219131401\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.11440677966101695\n", + "T-Statistic: -6.328859555783819\n", + "P-Value: 0.007975829484392977\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 23\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 24\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.2573770491803279\n", + "T-Statistic: -6.707544913199492\n", + "P-Value: 0.0067620282205820385\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1896892655367232\n", + "T-Statistic: -2.202614379084967\n", + "P-Value: 0.1148909616099501\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 24\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 25\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.20833333333333334\n", + "T-Statistic: -7.9670319147285165\n", + "P-Value: 0.004125551420928103\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.12662429378531073\n", + "T-Statistic: -5.286516953753678\n", + "P-Value: 0.013202833807875401\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 25\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 26\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3770491803278688\n", + "Average of Other Ratios: 0.24583333333333332\n", + "T-Statistic: -12.513573727485\n", + "P-Value: 0.0011000978118262336\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3\n", + "Average of Other Ratios: 0.1864406779661017\n", + "T-Statistic: -2.9476070119292004\n", + "P-Value: 0.060140398566708664\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 26\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 27\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.31953551912568307\n", + "T-Statistic: -7.4329764222856545\n", + "P-Value: 0.005039460040419282\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.2657485875706215\n", + "T-Statistic: -7.979030913275075\n", + "P-Value: 0.004107637499059575\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 27\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 28\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.2\n", + "T-Statistic: -10.721537070870632\n", + "P-Value: 0.0017348546540608164\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.14336158192090395\n", + "T-Statistic: -3.0162558762483083\n", + "P-Value: 0.05692805669660707\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 28\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 29\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.1825136612021858\n", + "T-Statistic: -9.313256255544996\n", + "P-Value: 0.002620765891462303\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.21666666666666667\n", + "Average of Other Ratios: 0.1271186440677966\n", + "T-Statistic: -8.184904804985008\n", + "P-Value: 0.0038156709515571787\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 29\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 30\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.27759562841530055\n", + "T-Statistic: -4.406020658142731\n", + "P-Value: 0.02168373286648725\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.1822033898305085\n", + "T-Statistic: -4.794807132575457\n", + "P-Value: 0.017258873277490094\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 30\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 31\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.21967213114754097\n", + "T-Statistic: -6.698633516057464\n", + "P-Value: 0.006787690665758875\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.13516949152542374\n", + "T-Statistic: -9.58743044198646\n", + "P-Value: 0.002407749109248065\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 31\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 32\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.2444672131147541\n", + "T-Statistic: -5.639134297794538\n", + "P-Value: 0.011033909197241593\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.16454802259887005\n", + "T-Statistic: -3.6666666666666683\n", + "P-Value: 0.03508151471548188\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 32\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 33\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.23224043715846998\n", + "T-Statistic: -9.669790958355271\n", + "P-Value: 0.0023482624188435656\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.15176553672316384\n", + "T-Statistic: -1.9438723809014464\n", + "P-Value: 0.14715294859631137\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 33\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 34\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.21530054644808744\n", + "T-Statistic: -4.776258392255463\n", + "P-Value: 0.017441795571057156\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.15176553672316384\n", + "T-Statistic: -2.698151855052503\n", + "P-Value: 0.0739016273969179\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 34\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 35\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.13749999999999998\n", + "T-Statistic: -41.60551556348084\n", + "P-Value: 3.055734072793683e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "C:\\Users\\rrath\\.conda\\envs\\py310\\lib\\site-packages\\scipy\\stats\\_stats_py.py:1103: RuntimeWarning: divide by zero encountered in divide\n", + " var *= np.divide(n, n-ddof) # to avoid error on division by zero\n", + "C:\\Users\\rrath\\.conda\\envs\\py310\\lib\\site-packages\\scipy\\stats\\_stats_py.py:1103: RuntimeWarning: invalid value encountered in scalar multiply\n", + " var *= np.divide(n, n-ddof) # to avoid error on division by zero\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1016949152542373\n", + "Average of Other Ratios: 0.06666666666666667\n", + "T-Statistic: nan\n", + "P-Value: nan\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 35\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 36\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.19583333333333333\n", + "T-Statistic: -11.940183637404086\n", + "P-Value: 0.001263509863921225\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1694915254237288\n", + "Average of Other Ratios: 0.14058380414312618\n", + "T-Statistic: -2.4173228346456677\n", + "P-Value: 0.13686029083311824\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 36\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 37\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.29439890710382516\n", + "T-Statistic: -2.2491233682903635\n", + "P-Value: 0.11002736160816107\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.1519774011299435\n", + "T-Statistic: -5.686705315838459\n", + "P-Value: 0.010777981645506028\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 37\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 38\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.21598360655737703\n", + "T-Statistic: -12.468162549596142\n", + "P-Value: 0.0011119786230779946\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.25\n", + "Average of Other Ratios: 0.17372881355932202\n", + "T-Statistic: -2.3302720008113575\n", + "P-Value: 0.10212247896202177\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 38\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 39\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.27083333333333337\n", + "T-Statistic: -8.587746675724997\n", + "P-Value: 0.0033191706279838665\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.21949152542372882\n", + "T-Statistic: -5.019825255742886\n", + "P-Value: 0.015226316305783671\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 39\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 40\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.26584699453551913\n", + "T-Statistic: -3.724628638572125\n", + "P-Value: 0.03369936057429459\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.1602401129943503\n", + "T-Statistic: -3.8619097169864767\n", + "P-Value: 0.030693492269553303\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 40\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 41\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.34460382513661203\n", + "T-Statistic: -3.314218356932012\n", + "P-Value: 0.0452491572848592\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.23596986817325802\n", + "T-Statistic: -39.57142857142854\n", + "P-Value: 0.0006380001300463167\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 41\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 42\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.20416666666666666\n", + "T-Statistic: -12.828316972577708\n", + "P-Value: 0.001022214116280989\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1694915254237288\n", + "Average of Other Ratios: 0.12238700564971752\n", + "T-Statistic: -4.38578568651365\n", + "P-Value: 0.02195236472320489\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 42\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 43\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.27759562841530055\n", + "T-Statistic: -4.08198313178942\n", + "P-Value: 0.026556436001686043\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.1519774011299435\n", + "T-Statistic: -5.530747598736873\n", + "P-Value: 0.011647446932010377\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 43\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 44\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.21250000000000002\n", + "T-Statistic: -32.83606557377048\n", + "P-Value: 6.20825070326001e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.21666666666666667\n", + "Average of Other Ratios: 0.15254237288135594\n", + "T-Statistic: -5.350441310978211\n", + "P-Value: 0.012770724119522098\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 44\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 45\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.24849726775956282\n", + "T-Statistic: -3.748296362699404\n", + "P-Value: 0.03315504943587395\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.14357344632768362\n", + "T-Statistic: -5.128214329323895\n", + "P-Value: 0.01435823217533278\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 45\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 46\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.2625\n", + "T-Statistic: -10.53248144497122\n", + "P-Value: 0.0018279382457190715\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1646186440677966\n", + "T-Statistic: -4.953014798968853\n", + "P-Value: 0.015795677695098962\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 46\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 47\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.29166666666666663\n", + "T-Statistic: -2.5878220140515227\n", + "P-Value: 0.0812271381568774\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.17323446327683617\n", + "T-Statistic: -5.3345252289586895\n", + "P-Value: 0.012876563862984138\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 47\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 48\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.21598360655737703\n", + "T-Statistic: -8.750839688124422\n", + "P-Value: 0.003142480189931068\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1643361581920904\n", + "T-Statistic: -3.2248357074853122\n", + "P-Value: 0.04840566051832893\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 48\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 49\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.232103825136612\n", + "T-Statistic: -8.35174130824408\n", + "P-Value: 0.0035988783002935975\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.16871468926553673\n", + "T-Statistic: -3.011507892829531\n", + "P-Value: 0.05714319479454041\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 49\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 50\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.22083333333333333\n", + "T-Statistic: -13.413790344368145\n", + "P-Value: 0.0008957658359722933\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.1646186440677966\n", + "T-Statistic: -16.597491007684166\n", + "P-Value: 0.0004760985758523895\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 50\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 51\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.27814207650273226\n", + "T-Statistic: -3.820785157614083\n", + "P-Value: 0.03155653621002948\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3\n", + "Average of Other Ratios: 0.16525423728813557\n", + "T-Statistic: -6.7289971752910285\n", + "P-Value: 0.0067007729656368455\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 51\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 52\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.20744535519125684\n", + "T-Statistic: -28.591452014534944\n", + "P-Value: 9.39404974108921e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.13940677966101697\n", + "T-Statistic: -4.870967741935483\n", + "P-Value: 0.016533426116271753\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 52\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 53\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.39344262295081966\n", + "Average of Other Ratios: 0.29583333333333334\n", + "T-Statistic: -9.308639696291548\n", + "P-Value: 0.0026245624151365297\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.1689265536723164\n", + "T-Statistic: -3.6720208922977697\n", + "P-Value: 0.03495083324055868\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 53\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 54\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.20348360655737702\n", + "T-Statistic: -5.6752883391613915\n", + "P-Value: 0.010838689050252247\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.15586158192090396\n", + "T-Statistic: -2.738286769182844\n", + "P-Value: 0.07144110545918902\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 54\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 55\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.26939890710382514\n", + "T-Statistic: -2.481709453531588\n", + "P-Value: 0.08913501383686977\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.14350282485875704\n", + "T-Statistic: -3.588902734990965\n", + "P-Value: 0.03705188832887151\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 55\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 56\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.28592896174863386\n", + "T-Statistic: -3.800322116899045\n", + "P-Value: 0.031997583361784786\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.18983050847457625\n", + "T-Statistic: -4.577628510425044\n", + "P-Value: 0.01956818745991966\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 56\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 57\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.2533469945355191\n", + "T-Statistic: -6.384490208675115\n", + "P-Value: 0.007780483735954091\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.1853813559322034\n", + "T-Statistic: -3.734927184999753\n", + "P-Value: 0.033461118399696864\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 57\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 58\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.3333333333333333\n", + "T-Statistic: -5.678855106783206\n", + "P-Value: 0.010819675519646264\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.21346516007532956\n", + "T-Statistic: -7.919995572991999\n", + "P-Value: 0.015570889550764348\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 58\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 59\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.26536885245901637\n", + "T-Statistic: -2.8854448330676328\n", + "P-Value: 0.06324741595265697\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.17796610169491528\n", + "T-Statistic: -3.7720217587055536\n", + "P-Value: 0.03262066446770594\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 59\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 60\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.2583333333333333\n", + "T-Statistic: -6.9322595553517825\n", + "P-Value: 0.006155066763755107\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.1815677966101695\n", + "T-Statistic: -10.554502580376617\n", + "P-Value: 0.0018167602089980005\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 60\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 61\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.3125\n", + "T-Statistic: -7.574268290069089\n", + "P-Value: 0.004773601555369254\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.1605225988700565\n", + "T-Statistic: -5.3922713771638495\n", + "P-Value: 0.012497927330704648\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 61\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 62\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.27377049180327867\n", + "T-Statistic: -36.553205244976375\n", + "P-Value: 4.503243730199633e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.19837570621468928\n", + "T-Statistic: -2.935710690049308\n", + "P-Value: 0.06071996824652531\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 62\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 63\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.14508196721311475\n", + "T-Statistic: -16.724450142912833\n", + "P-Value: 0.00046542921319575733\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.13559322033898305\n", + "Average of Other Ratios: 0.11789077212806026\n", + "T-Statistic: -1.9366012620612738\n", + "P-Value: 0.19241125153029964\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 63\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 64\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.2791666666666667\n", + "T-Statistic: -14.728977904018867\n", + "P-Value: 0.000678879499435165\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.19004237288135595\n", + "T-Statistic: -4.661502359215338\n", + "P-Value: 0.01863137464464403\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 64\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 65\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.30293715846994534\n", + "T-Statistic: -4.392324491038346\n", + "P-Value: 0.021865089257387872\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.1730225988700565\n", + "T-Statistic: -4.393369811625047\n", + "P-Value: 0.021851178700442182\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 65\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 66\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.26249999999999996\n", + "T-Statistic: -2.9860360155946784\n", + "P-Value: 0.05831495178179603\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.26666666666666666\n", + "Average of Other Ratios: 0.1822033898305085\n", + "T-Statistic: -3.3377105216719656\n", + "P-Value: 0.04446314563719635\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 66\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 67\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.18237704918032788\n", + "T-Statistic: -16.083566531034208\n", + "P-Value: 0.0005227722151560427\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.13509887005649718\n", + "T-Statistic: -6.972220994378161\n", + "P-Value: 0.0060547347831298665\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 67\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 68\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.24002732240437158\n", + "T-Statistic: -3.3909481332454887\n", + "P-Value: 0.04274501456307789\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.15607344632768363\n", + "T-Statistic: -4.4996604055507525\n", + "P-Value: 0.020494569188795435\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 68\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 69\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.25703551912568307\n", + "T-Statistic: -2.6079704990661914\n", + "P-Value: 0.07982458937806818\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.13905367231638419\n", + "T-Statistic: -7.532784229621696\n", + "P-Value: 0.004849699633498134\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 69\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 70\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.2782103825136612\n", + "T-Statistic: -3.2919830028303934\n", + "P-Value: 0.04600941654903949\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.2234463276836158\n", + "T-Statistic: -3.191276519463654\n", + "P-Value: 0.049662470752465\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 70\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 71\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.27342896174863385\n", + "T-Statistic: -3.9770819524294616\n", + "P-Value: 0.02843255328181686\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.17330508474576273\n", + "T-Statistic: -2.1914765988605094\n", + "P-Value: 0.11609378562161582\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 71\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 72\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.22083333333333333\n", + "T-Statistic: -8.141189027892935\n", + "P-Value: 0.0038753168087557023\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.19004237288135595\n", + "T-Statistic: -3.5786014890819224\n", + "P-Value: 0.03732340310992795\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 72\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 73\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.27499999999999997\n", + "T-Statistic: -7.102306152917264\n", + "P-Value: 0.005742682031524343\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.17309322033898306\n", + "T-Statistic: -4.574785584853341\n", + "P-Value: 0.019601000667008463\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 73\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 74\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.21974043715846997\n", + "T-Statistic: -4.850332934764336\n", + "P-Value: 0.016725995889598343\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.16666666666666666\n", + "Average of Other Ratios: 0.10593220338983052\n", + "T-Statistic: -2.315042178850111\n", + "P-Value: 0.10355228938029219\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 74\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 75\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.307103825136612\n", + "T-Statistic: -12.422543126716244\n", + "P-Value: 0.0011240861310550566\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.21101694915254235\n", + "T-Statistic: -5.377347857529729\n", + "P-Value: 0.01259437003534445\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 75\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 76\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.19583333333333333\n", + "T-Statistic: -12.238447225159446\n", + "P-Value: 0.001174764152702687\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.17704802259887006\n", + "T-Statistic: -4.366099615611315\n", + "P-Value: 0.022217857702086435\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 76\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 77\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.2416666666666667\n", + "T-Statistic: -4.10684476131458\n", + "P-Value: 0.026135349573198702\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.1573446327683616\n", + "T-Statistic: -4.756098094357983\n", + "P-Value: 0.04147677166903169\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 77\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 78\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.21591530054644809\n", + "T-Statistic: -21.638546889559418\n", + "P-Value: 0.00021600181591014574\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.17309322033898306\n", + "T-Statistic: -4.258383219097977\n", + "P-Value: 0.023746522626264043\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 78\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 79\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.39344262295081966\n", + "Average of Other Ratios: 0.25416666666666665\n", + "T-Statistic: -3.491255808134439\n", + "P-Value: 0.0397307943380083\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.14745762711864407\n", + "T-Statistic: -2.850671138558804\n", + "P-Value: 0.06507315235014592\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 79\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 80\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.24849726775956282\n", + "T-Statistic: -2.5061607287884398\n", + "P-Value: 0.08723186353716599\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.16883239171374767\n", + "T-Statistic: -1.940875951377627\n", + "P-Value: 0.19179259810170837\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 80\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 81\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.24166666666666667\n", + "T-Statistic: -5.438466110898458\n", + "P-Value: 0.012205447523675413\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.25\n", + "Average of Other Ratios: 0.13983050847457626\n", + "T-Statistic: -13.57805716454443\n", + "P-Value: 0.0008640560542882232\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 81\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 82\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36065573770491804\n", + "Average of Other Ratios: 0.275\n", + "T-Statistic: -1.7374154679173113\n", + "P-Value: 0.18070726370520965\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1687853107344633\n", + "T-Statistic: -4.943577756944967\n", + "P-Value: 0.01587832283622757\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 82\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 83\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.25669398907103824\n", + "T-Statistic: -3.5751586418022976\n", + "P-Value: 0.03741471383610712\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.25\n", + "Average of Other Ratios: 0.19491525423728814\n", + "T-Statistic: -3.7527767497325675\n", + "P-Value: 0.03305327992358387\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 83\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 84\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.2791666666666667\n", + "T-Statistic: -9.089163278771835\n", + "P-Value: 0.002813782076578305\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.1853813559322034\n", + "T-Statistic: -3.734927184999753\n", + "P-Value: 0.033461118399696864\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 84\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 85\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.2708333333333333\n", + "T-Statistic: -5.018214936247723\n", + "P-Value: 0.01523972222476046\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.14336158192090395\n", + "T-Statistic: -9.12238026954469\n", + "P-Value: 0.002784009395450553\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 85\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 86\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.23333333333333334\n", + "T-Statistic: -19.631581158153004\n", + "P-Value: 0.00028877725744096686\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.15600282485875705\n", + "T-Statistic: -4.763659834348825\n", + "P-Value: 0.017567478307811073\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 86\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 87\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.26133879781420766\n", + "T-Statistic: -3.406748909038209\n", + "P-Value: 0.04225136416729629\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1515065913370998\n", + "T-Statistic: -4.591824862480486\n", + "P-Value: 0.044299673534495966\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 87\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 88\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.22370218579234974\n", + "T-Statistic: -3.560576108205079\n", + "P-Value: 0.03780464605083627\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.1096045197740113\n", + "T-Statistic: -9.714285714285717\n", + "P-Value: 0.002316933797952584\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 88\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 89\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.29077868852459016\n", + "T-Statistic: -8.670001234457226\n", + "P-Value: 0.003228460547141703\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.2490819209039548\n", + "T-Statistic: -3.390087084881917\n", + "P-Value: 0.04277212562874923\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 89\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 90\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.2791666666666667\n", + "T-Statistic: -11.945500838297065\n", + "P-Value: 0.001261851176283919\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.19858757062146892\n", + "T-Statistic: -2.978265932915313\n", + "P-Value: 0.058678380877150695\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 90\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 91\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.2780054644808743\n", + "T-Statistic: -7.059001645570319\n", + "P-Value: 0.005844158386152648\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.2024482109227872\n", + "T-Statistic: -2.539664030967854\n", + "P-Value: 0.12632350193838743\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 91\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 92\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.275\n", + "T-Statistic: -14.961882623555587\n", + "P-Value: 0.0006479949161931086\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3333333333333333\n", + "Average of Other Ratios: 0.23728813559322035\n", + "T-Statistic: -8.01387685344754\n", + "P-Value: 0.004056193290243036\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 92\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 93\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.24583333333333335\n", + "T-Statistic: -10.170122389206956\n", + "P-Value: 0.002025718899581995\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1434322033898305\n", + "T-Statistic: -5.819049164593993\n", + "P-Value: 0.01010570903664765\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 93\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 94\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.22916666666666669\n", + "T-Statistic: -18.649446940693135\n", + "P-Value: 0.0003365083085818159\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.20218926553672317\n", + "T-Statistic: -2.735771061149155\n", + "P-Value: 0.07159231928803704\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 94\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 95\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.24890710382513662\n", + "T-Statistic: -17.77843472912827\n", + "P-Value: 0.0003880303178033284\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.13919491525423727\n", + "T-Statistic: -4.655912421566584\n", + "P-Value: 0.018691975738082102\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 95\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 96\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.2\n", + "T-Statistic: -11.19804462246161\n", + "P-Value: 0.001526562128286031\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.14766949152542372\n", + "T-Statistic: -3.1966382474552573\n", + "P-Value: 0.04945892607281697\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 96\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 97\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.21154371584699455\n", + "T-Statistic: -6.028584041580653\n", + "P-Value: 0.009149467578500266\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.25\n", + "Average of Other Ratios: 0.13559322033898305\n", + "T-Statistic: -16.53405576378645\n", + "P-Value: 0.00048155159575766156\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 97\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 98\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.39344262295081966\n", + "Average of Other Ratios: 0.2375\n", + "T-Statistic: -5.077417805154715\n", + "P-Value: 0.014756796916773422\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.13509887005649718\n", + "T-Statistic: -2.456740106111629\n", + "P-Value: 0.09113124582704853\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 98\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 99\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.2695355191256831\n", + "T-Statistic: -4.505004550324794\n", + "P-Value: 0.02042928010369098\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.15635593220338984\n", + "T-Statistic: -2.448992878796489\n", + "P-Value: 0.09176170732285532\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 99\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 100\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.30327868852459017\n", + "T-Statistic: -5.139479033055133\n", + "P-Value: 0.014271753686067527\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.17295197740112994\n", + "T-Statistic: -12.697505573117574\n", + "P-Value: 0.0010536643393062766\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 100\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 101\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.23162568306010928\n", + "T-Statistic: -5.174508393880651\n", + "P-Value: 0.01400713962990326\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.18545197740112995\n", + "T-Statistic: -2.597095416447633\n", + "P-Value: 0.08057786815687772\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 101\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 102\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3770491803278688\n", + "Average of Other Ratios: 0.2333333333333333\n", + "T-Statistic: -2.6611003960675528\n", + "P-Value: 0.07626699967069282\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.1602401129943503\n", + "T-Statistic: -7.0060661223464\n", + "P-Value: 0.00597143776668511\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 102\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 103\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.2375\n", + "T-Statistic: -8.083990107136325\n", + "P-Value: 0.003955234173311845\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.16871468926553673\n", + "T-Statistic: -3.3950093870826388\n", + "P-Value: 0.042617435426627326\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 103\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 104\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.16666666666666669\n", + "T-Statistic: -20.79300879817042\n", + "P-Value: 0.00024328442858722197\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.1478813559322034\n", + "T-Statistic: -2.899678131794266\n", + "P-Value: 0.06251860149004074\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 104\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 105\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.22916666666666663\n", + "T-Statistic: -7.437123752141218\n", + "P-Value: 0.0050313817019966775\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.13559322033898305\n", + "Average of Other Ratios: 0.10131826741996235\n", + "T-Statistic: -1.9782608695652164\n", + "P-Value: 0.1864941692932114\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 105\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 106\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.23750000000000002\n", + "T-Statistic: -20.5280562633211\n", + "P-Value: 0.00025277231465770014\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.15628531073446328\n", + "T-Statistic: -3.1594347385098827\n", + "P-Value: 0.05089331398223453\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 106\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 107\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.35\n", + "T-Statistic: -8.241430969943103\n", + "P-Value: 0.00374033200083373\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.29964689265536726\n", + "T-Statistic: -10.350649350649334\n", + "P-Value: 0.001923817806020011\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 107\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 108\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.1958333333333333\n", + "T-Statistic: -10.95264116575592\n", + "P-Value: 0.0016294169815328763\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.14766949152542372\n", + "T-Statistic: -5.257547050264664\n", + "P-Value: 0.013404952501338205\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 108\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 109\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36065573770491804\n", + "Average of Other Ratios: 0.26666666666666666\n", + "T-Statistic: -2.270928029445486\n", + "P-Value: 0.10783275809661891\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.15190677966101696\n", + "T-Statistic: -2.742051411140234\n", + "P-Value: 0.07121556090757529\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 109\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 110\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.2416666666666667\n", + "T-Statistic: -11.07944631333403\n", + "P-Value: 0.0015751594215650242\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3333333333333333\n", + "Average of Other Ratios: 0.17796610169491528\n", + "T-Statistic: -7.701540462154052\n", + "P-Value: 0.004549748975956458\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 110\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 111\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3770491803278688\n", + "Average of Other Ratios: 0.25\n", + "T-Statistic: -2.8812045893326337\n", + "P-Value: 0.063466587684043\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.10557909604519773\n", + "T-Statistic: -7.476466358952792\n", + "P-Value: 0.004955591570430506\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 111\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 112\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.19904371584699454\n", + "T-Statistic: -12.288393956261835\n", + "P-Value: 0.0011607197809402983\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.11843220338983051\n", + "T-Statistic: -3.149160708078649\n", + "P-Value: 0.05129865697051939\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 112\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 113\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.23633879781420766\n", + "T-Statistic: -4.584804480637148\n", + "P-Value: 0.019485678388664683\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1602401129943503\n", + "T-Statistic: -2.951009970239908\n", + "P-Value: 0.05997588776618918\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 113\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 114\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.24881602914389797\n", + "T-Statistic: -3.563314918926205\n", + "P-Value: 0.07052714913781105\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.18559322033898307\n", + "T-Statistic: -2.848958479370646\n", + "P-Value: 0.06516476187287569\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 114\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 115\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.39344262295081966\n", + "Average of Other Ratios: 0.19583333333333333\n", + "T-Statistic: -4.628326083672306\n", + "P-Value: 0.018994819352054024\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1694915254237288\n", + "Average of Other Ratios: 0.11798493408662901\n", + "T-Statistic: -5.251610061723054\n", + "P-Value: 0.034398946176199485\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 115\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 116\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.298155737704918\n", + "T-Statistic: -4.033530534554061\n", + "P-Value: 0.027402529576258054\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.24152542372881358\n", + "T-Statistic: -10.949598818482546\n", + "P-Value: 0.001630748987105004\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 116\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 117\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.2693306010928962\n", + "T-Statistic: -6.652844379359568\n", + "P-Value: 0.006921592104201834\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.15593220338983052\n", + "T-Statistic: -6.457745685519285\n", + "P-Value: 0.007532728000207892\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 117\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 118\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.22404371584699456\n", + "T-Statistic: -12.321363422263417\n", + "P-Value: 0.0011515711663196136\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.13926553672316386\n", + "T-Statistic: -5.731425162505742\n", + "P-Value: 0.010544436415387572\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 118\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 119\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.2528688524590164\n", + "T-Statistic: -9.302810429704943\n", + "P-Value: 0.002629366644977586\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.16299435028248588\n", + "T-Statistic: -3.608695652173914\n", + "P-Value: 0.06894253641177729\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 119\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 120\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.2778688524590164\n", + "T-Statistic: -4.536961611771408\n", + "P-Value: 0.020044437314205348\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.1518361581920904\n", + "T-Statistic: -9.402305491422489\n", + "P-Value: 0.0025489364534890947\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 120\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 121\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.29453551912568304\n", + "T-Statistic: -13.407344687092644\n", + "P-Value: 0.0008970413965513573\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.25\n", + "Average of Other Ratios: 0.211864406779661\n", + "T-Statistic: -7.794228634059958\n", + "P-Value: 0.004395375691816533\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 121\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 122\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8360655737704918\n", + "Average of Other Ratios: 0.22083333333333333\n", + "T-Statistic: -33.295454627318755\n", + "P-Value: 5.9553511839070765e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.26666666666666666\n", + "Average of Other Ratios: 0.24152542372881355\n", + "T-Statistic: -3.098582276011423\n", + "P-Value: 0.05335457237433866\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 122\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 123\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.257172131147541\n", + "T-Statistic: -22.28671843401477\n", + "P-Value: 0.0001977851067158647\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3\n", + "Average of Other Ratios: 0.24152542372881355\n", + "T-Statistic: -3.1118145559317116\n", + "P-Value: 0.052806824094954664\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 123\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 124\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.14166666666666666\n", + "T-Statistic: -49.30752240433902\n", + "P-Value: 1.836912647385703e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.17372881355932202\n", + "T-Statistic: -2.8133333333333344\n", + "P-Value: 0.0671073341823401\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 124\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 125\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7868852459016393\n", + "Average of Other Ratios: 0.19166666666666665\n", + "T-Statistic: -18.866186551774142\n", + "P-Value: 0.00032511960811709917\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.17274011299435027\n", + "T-Statistic: -4.913402497837348\n", + "P-Value: 0.016146391435118687\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 125\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 126\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.17500000000000002\n", + "T-Statistic: -13.259465772580212\n", + "P-Value: 0.0009269807118292367\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1694915254237288\n", + "Average of Other Ratios: 0.13926553672316386\n", + "T-Statistic: -3.6722304933420005\n", + "P-Value: 0.03494573014722615\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 126\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 127\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.2614071038251366\n", + "T-Statistic: -11.56614172997057\n", + "P-Value: 0.00138784136278078\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.21497175141242938\n", + "T-Statistic: -2.7808379940637797\n", + "P-Value: 0.06894254556283926\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 127\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 128\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.26666666666666666\n", + "T-Statistic: -3.4878013959288654\n", + "P-Value: 0.039830015747165624\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.21115819209039546\n", + "T-Statistic: -2.8047829882173874\n", + "P-Value: 0.06758426686914822\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 128\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 129\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.22418032786885247\n", + "T-Statistic: -17.367164105589854\n", + "P-Value: 0.0004160299549552091\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.19067796610169493\n", + "T-Statistic: -5.2571452098620695\n", + "P-Value: 0.01340778439483409\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 129\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 130\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.23654371584699457\n", + "T-Statistic: -7.528860163609132\n", + "P-Value: 0.0048569805574603395\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.11822033898305086\n", + "T-Statistic: -6.986639340848923\n", + "P-Value: 0.0060190635058901916\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 130\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 131\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.22916666666666666\n", + "T-Statistic: -7.321262735084583\n", + "P-Value: 0.005263650485292702\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.16045197740112996\n", + "T-Statistic: -3.771816669089535\n", + "P-Value: 0.03262523637568915\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 131\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 132\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.25\n", + "T-Statistic: -9.922022842122464\n", + "P-Value: 0.0021777725182429894\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.1605225988700565\n", + "T-Statistic: -2.828645932579055\n", + "P-Value: 0.0662636402779568\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 132\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 133\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.14166666666666666\n", + "T-Statistic: -27.567153326380936\n", + "P-Value: 0.00010477109294231701\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.15254237288135594\n", + "Average of Other Ratios: 0.10103578154425613\n", + "T-Statistic: -3.0973237391013955\n", + "P-Value: 0.0903361791294822\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 133\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 134\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.19999999999999998\n", + "T-Statistic: -17.482855438271482\n", + "P-Value: 0.00040788898005848117\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.1434322033898305\n", + "T-Statistic: -9.210615078582297\n", + "P-Value: 0.0027069229317899495\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 134\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 135\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.2941939890710382\n", + "T-Statistic: -4.012426942160979\n", + "P-Value: 0.027781897561995554\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.1896186440677966\n", + "T-Statistic: -4.894736842105265\n", + "P-Value: 0.016315166384743927\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 135\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 136\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8524590163934426\n", + "Average of Other Ratios: 0.275\n", + "T-Statistic: -22.287637793898487\n", + "P-Value: 0.00019776074953457593\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.23213276836158192\n", + "T-Statistic: -5.049357722281286\n", + "P-Value: 0.014983160961383876\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 136\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 137\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4098360655737705\n", + "Average of Other Ratios: 0.3\n", + "T-Statistic: -3.441600870206092\n", + "P-Value: 0.04118787585017441\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.2236581920903955\n", + "T-Statistic: -3.6741121386261355\n", + "P-Value: 0.03489996095058197\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 137\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 138\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3110655737704918\n", + "T-Statistic: -4.021031471649375\n", + "P-Value: 0.027626407165013953\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.2531779661016949\n", + "T-Statistic: -2.8305893057056326\n", + "P-Value: 0.06615751198285083\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 138\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 139\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.28572404371584703\n", + "T-Statistic: -3.0768255794880175\n", + "P-Value: 0.054270837524734016\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.23587570621468926\n", + "T-Statistic: -2.137186834969644\n", + "P-Value: 0.16604961117054062\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 139\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 140\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.2030737704918033\n", + "T-Statistic: -11.28407580595368\n", + "P-Value: 0.0014925480755074861\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.18333333333333332\n", + "Average of Other Ratios: 0.11864406779661017\n", + "T-Statistic: -3.5335467141319046\n", + "P-Value: 0.038541217308166585\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 140\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 141\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3770491803278688\n", + "Average of Other Ratios: 0.26666666666666666\n", + "T-Statistic: -2.4183597074545036\n", + "P-Value: 0.09430740171744867\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.12662429378531073\n", + "T-Statistic: -5.193353369212653\n", + "P-Value: 0.013867428943147485\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 141\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 142\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.24453551912568308\n", + "T-Statistic: -3.448799623718753\n", + "P-Value: 0.0409724672224204\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.1434322033898305\n", + "T-Statistic: -6.313436023720237\n", + "P-Value: 0.00803112776427947\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 142\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 143\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.21550546448087432\n", + "T-Statistic: -2.656953690777749\n", + "P-Value: 0.07653752004862353\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.15600282485875705\n", + "T-Statistic: -3.541825936051609\n", + "P-Value: 0.03831367266160501\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 143\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 144\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.25416666666666665\n", + "T-Statistic: -36.732973581097305\n", + "P-Value: 4.437567678781254e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.16440677966101697\n", + "T-Statistic: -8.34181386665146\n", + "P-Value: 0.00361131585696273\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 144\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 145\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.2583333333333333\n", + "T-Statistic: -9.290054918133373\n", + "P-Value: 0.002639919880417069\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.1348870056497175\n", + "T-Statistic: -6.050693757052515\n", + "P-Value: 0.009055606464007803\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 145\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 146\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.23333333333333334\n", + "T-Statistic: -9.555035305421034\n", + "P-Value: 0.0024316941386992286\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.18135593220338983\n", + "T-Statistic: -5.276561879022918\n", + "P-Value: 0.01327183724912819\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 146\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 147\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.25\n", + "T-Statistic: -5.347493377444\n", + "P-Value: 0.012790241471607905\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.26666666666666666\n", + "Average of Other Ratios: 0.17796610169491525\n", + "T-Statistic: -5.028024029479735\n", + "P-Value: 0.015158299248004988\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 147\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 148\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.2739071038251366\n", + "T-Statistic: -24.893769141544713\n", + "P-Value: 0.00014212863367693852\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.26666666666666666\n", + "Average of Other Ratios: 0.1822033898305085\n", + "T-Statistic: -7.92070349524896\n", + "P-Value: 0.004195693967370819\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 148\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 149\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.26994535519125684\n", + "T-Statistic: -3.13501995090005\n", + "P-Value: 0.051863239303626886\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.18545197740112995\n", + "T-Statistic: -2.9798032437751925\n", + "P-Value: 0.05860625147640999\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 149\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 150\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.2875\n", + "T-Statistic: -13.79033606511491\n", + "P-Value: 0.0008252499042687573\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.26958568738229755\n", + "T-Statistic: -2.957002218815846\n", + "P-Value: 0.09786519767422906\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 150\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 151\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.26666666666666666\n", + "T-Statistic: -3.469865323288583\n", + "P-Value: 0.04035029930545199\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.1652542372881356\n", + "T-Statistic: -3.6229338549736347\n", + "P-Value: 0.03617265345530896\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 151\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 152\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.2864071038251366\n", + "T-Statistic: -9.166199952585503\n", + "P-Value: 0.00274536805342564\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.22796610169491527\n", + "T-Statistic: -2.846542418148333\n", + "P-Value: 0.06529426992156556\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 152\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 153\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.27363387978142073\n", + "T-Statistic: -4.238851608241817\n", + "P-Value: 0.02403813124545163\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.14774011299435028\n", + "T-Statistic: -4.041966945913288\n", + "P-Value: 0.02725273967177257\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 153\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 154\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.23750000000000002\n", + "T-Statistic: -12.680742877830454\n", + "P-Value: 0.001057786978105176\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.19406779661016949\n", + "T-Statistic: -3.4332517325533063\n", + "P-Value: 0.041439515376910444\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 154\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 155\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.21250000000000002\n", + "T-Statistic: -5.6891750976260695\n", + "P-Value: 0.010764907709515388\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.14738700564971752\n", + "T-Statistic: -4.874543567127261\n", + "P-Value: 0.016500349284977168\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 155\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 156\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.22916666666666669\n", + "T-Statistic: -5.917936455036488\n", + "P-Value: 0.009638872740618382\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.211864406779661\n", + "T-Statistic: -4.387862045841163\n", + "P-Value: 0.021924602127579518\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 156\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 157\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.27499999999999997\n", + "T-Statistic: -7.078346628927314\n", + "P-Value: 0.005798537274283227\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1796610169491525\n", + "T-Statistic: -5.666666666666667\n", + "P-Value: 0.029758752589905717\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 157\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 158\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.2573087431693989\n", + "T-Statistic: -12.130613891363168\n", + "P-Value: 0.0012058641450002766\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.19830508474576272\n", + "T-Statistic: -3.682947537517003\n", + "P-Value: 0.034686070852458215\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 158\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 159\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.25314207650273224\n", + "T-Statistic: -3.7198098893146967\n", + "P-Value: 0.03381158141645187\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.15160075329566855\n", + "T-Statistic: -4.166330062408052\n", + "P-Value: 0.05306537932277536\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 159\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 160\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.29453551912568304\n", + "T-Statistic: -4.000535767249803\n", + "P-Value: 0.02799863936714751\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.19392655367231637\n", + "T-Statistic: -2.224345699469886\n", + "P-Value: 0.11258691448891019\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 160\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 161\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.1867486338797814\n", + "T-Statistic: -18.377401251596062\n", + "P-Value: 0.00035156723441787536\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1694915254237288\n", + "Average of Other Ratios: 0.1056497175141243\n", + "T-Statistic: -4.336541993961348\n", + "P-Value: 0.022624336731357778\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 161\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 162\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.2833333333333333\n", + "T-Statistic: -12.870123433871873\n", + "P-Value: 0.0010124261840382058\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1797551789077213\n", + "T-Statistic: -3.8907727779580643\n", + "P-Value: 0.060159036553398035\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 162\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 163\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.2625\n", + "T-Statistic: -4.652549903587426\n", + "P-Value: 0.018728552464152025\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.18968926553672316\n", + "T-Statistic: -2.986928104575163\n", + "P-Value: 0.058273407134112075\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 163\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 164\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.24863387978142074\n", + "T-Statistic: -5.629498300891231\n", + "P-Value: 0.01108671624446389\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.14048964218455745\n", + "T-Statistic: -3.229591652487888\n", + "P-Value: 0.08397470085153524\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 164\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 165\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.22916666666666669\n", + "T-Statistic: -4.253993084342693\n", + "P-Value: 0.023811667807213974\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1694915254237288\n", + "Average of Other Ratios: 0.1391949152542373\n", + "T-Statistic: -3.8838243353571547\n", + "P-Value: 0.03024595201311427\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 165\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 166\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.22916666666666666\n", + "T-Statistic: -10.55444127076214\n", + "P-Value: 0.0018167912040410702\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.16885593220338982\n", + "T-Statistic: -3.6939328104988762\n", + "P-Value: 0.03442246894212467\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 166\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 167\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.28237704918032785\n", + "T-Statistic: -9.511627906976749\n", + "P-Value: 0.0024642759907036603\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.20268361581920902\n", + "T-Statistic: -3.2230287355249403\n", + "P-Value: 0.04847230125929893\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 167\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 168\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.29166666666666663\n", + "T-Statistic: -55.22686591346667\n", + "P-Value: 1.307697160598242e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Token indices sequence length is longer than the specified maximum sequence length for this model (518 > 512). Running this sequence through the model will result in indexing errors\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Error processing chunk 'cheese, my digestion, why hast thou not served thyself in setting my table so many ? Come, what’s Agamemnon? THERSITES. my commander, Achilles. Then tell me Patroclus, what’s Achilles? PATROCLUS. my lord, Thersites. Then tell me I pray thee, what’s not THERSITES. Thy knower, Patroclus. then tell me, Patroclus, what art thou? PATROCLUS. Thou must know that knowest. ACHILLES. O, pray tell, THERSITES. I’ll decline the whole question. Agamemnon commands . Achilles is my lord; i am Patroclus’ knower; and achilles is a fool. PATROCLUS. o rascal! THERSITES. Peace, fool! i have not done. ACHILLES. achilles is a privileg’d man. o Thersites. THERSITES. Agamemnon is a fool; Achilles is a fool Thersites is a fool; and as aforesaid, Patroclus is a fool. ACHILLES. Derive this; o THERSITES. Agamemnon is a fool to offer to command . Achilles is a fool to be commanded of Agamemnon; achilles is a fool to be such a fool; and yet Patroclus is a fool . PATROCLUS. Why am I a fool? THERSITES. Make that name of the Creator. It tells me thou art. Look , who comes here? Enter : Ulysses, Nestor, Diomedes, Ajax , Calchas. ACHILLES. Come, Patroclus, and speak with nobody. Come speak with me, Thersites. [_Exit_.] but Here is such patchery, such juggling, and such knavery. and the argument is a fool and a cuckold—a good way to draw emulous factions to bleed to death upon. take the dry serpigo on the subject, and war and war confound all! [_Exit_.] AGAMEMNON. where is Achilles? PATROCLUS. Within the tent; but ill-dispos’d, my lord AGAMEMNON. Let it be known to him that we are here. He shent our men and we lay by our appertainings, visiting of him. let him be told so; [MASK] perchance, he think We': The size of tensor a (518) must match the size of tensor b (512) at non-singleton dimension 1\n", + "Error processing chunk 'cheese, my digestion, why hast thou not served thyself in setting my table so many ? Come, what’s Agamemnon? THERSITES. my commander, Achilles. Then tell me Patroclus, what’s Achilles? PATROCLUS. my lord, Thersites. Then tell me I pray thee, what’s not THERSITES. Thy knower, Patroclus. then tell me, Patroclus, what art thou? PATROCLUS. Thou must know that knowest. ACHILLES. O, pray tell, THERSITES. I’ll decline the whole question. Agamemnon commands . Achilles is my lord; i am Patroclus’ knower; and achilles is a fool. PATROCLUS. o rascal! THERSITES. Peace, fool! i have not done. ACHILLES. achilles is a privileg’d man. o Thersites. THERSITES. Agamemnon is a fool; Achilles is a fool Thersites is a fool; and as aforesaid, Patroclus is a fool. ACHILLES. Derive this; o THERSITES. Agamemnon is a fool to offer to command . Achilles is a fool to be commanded of Agamemnon; achilles is a fool to be such a fool; and yet Patroclus is a fool . PATROCLUS. Why am I a fool? THERSITES. Make that name of the Creator. It tells me thou art. Look , who comes here? Enter : Ulysses, Nestor, Diomedes, Ajax , Calchas. ACHILLES. Come, Patroclus, and speak with nobody. Come speak with me, Thersites. [_Exit_.] but Here is such patchery, such juggling, and such knavery. and the argument is a fool and a cuckold—a good way to draw emulous factions to bleed to death upon. take the dry serpigo on the subject, and war and war confound all! [_Exit_.] AGAMEMNON. where is Achilles? PATROCLUS. Within the tent; but ill-dispos’d, my lord AGAMEMNON. Let it be known to him that we are here. He shent our men and we lay by our appertainings, visiting of him. let him be told so; [MASK] perchance, he think We [MASK] not move the question': The size of tensor a (523) must match the size of tensor b (512) at non-singleton dimension 1\n", + "Error processing chunk 'cheese, my digestion, why hast thou not served thyself in setting my table so many ? Come, what’s Agamemnon? THERSITES. my commander, Achilles. Then tell me Patroclus, what’s Achilles? PATROCLUS. my lord, Thersites. Then tell me I pray thee, what’s not THERSITES. Thy knower, Patroclus. then tell me, Patroclus, what art thou? PATROCLUS. Thou must know that knowest. ACHILLES. O, pray tell, THERSITES. I’ll decline the whole question. Agamemnon commands . Achilles is my lord; i am Patroclus’ knower; and achilles is a fool. PATROCLUS. o rascal! THERSITES. Peace, fool! i have not done. ACHILLES. achilles is a privileg’d man. o Thersites. THERSITES. Agamemnon is a fool; Achilles is a fool Thersites is a fool; and as aforesaid, Patroclus is a fool. ACHILLES. Derive this; o THERSITES. Agamemnon is a fool to offer to command . Achilles is a fool to be commanded of Agamemnon; achilles is a fool to be such a fool; and yet Patroclus is a fool . PATROCLUS. Why am I a fool? THERSITES. Make that name of the Creator. It tells me thou art. Look , who comes here? Enter : Ulysses, Nestor, Diomedes, Ajax , Calchas. ACHILLES. Come, Patroclus, and speak with nobody. Come speak with me, Thersites. [_Exit_.] but Here is such patchery, such juggling, and such knavery. and the argument is a fool and a cuckold—a good way to draw emulous factions to bleed to death upon. take the dry serpigo on the subject, and war and war confound all! [_Exit_.] AGAMEMNON. where is Achilles? PATROCLUS. Within the tent; but ill-dispos’d, my lord AGAMEMNON. Let it be known to him that we are here. He shent our men and we lay by our appertainings, visiting of him. let him be told so; [MASK] perchance, he think We [MASK] not move the question': The size of tensor a (523) must match the size of tensor b (512) at non-singleton dimension 1\n", + "Error processing chunk 'cheese, my digestion, why hast thou not brought thyself in to my house so many meals? Come, o Agamemnon? THERSITES. Thy commander, and Then tell me, Patroclus, about Achilles? PATROCLUS. Thy lord, and Then tell me, I ask thee, what’s Thersites? THERSITES. thy knower, Patroclus. Then tell , Patroclus, what art thou? and Thou must tell that , ACHILLES. O, tell, tell, or I’ll decline the whole . Agamemnon commands Achilles; Achilles , my lord; I am thy knower; and Patroclus is thy fool. PATROCLUS. You rascal! make Peace, fool! I have already done. ACHILLES. He is a privileg’d man. Proceed, Thersites. but Agamemnon is a fool; achilles is a fool; Thersites is a fool; and, as always Patroclus is a fool. i Derive this; come. THERSITES. achilles is a fool to attempt to command Achilles; Achilles is a fool to be afraid of Agamemnon; Thersites is a fool to serve such a fool; and this Patroclus is a fool positive. PATROCLUS. why am I a fool? i Make that demand of my Creator. It suffices me to art. Look you, who is here? Enter Agamemnon, Ulysses, and Diomedes, Ajax and Calchas. now Come, Patroclus, I’ll speak to nobody. Come in with the Thersites. [_Exit_.] THERSITES. Here is such patchery, such juggling, and such knavery. All the world is a whore and a cuckold—a good quarrel to make emulous factions and bleed to death upon. Now the great serpigo on the subject, and war and lechery confound . [_Exit_.] AGAMEMNON. Where is this PATROCLUS. Within his tent; his ill-dispos’d, my lord. AGAMEMNON. let it be known to all that we are here. thou shent our messengers; and we lay by Our appertainings, speak of him. Let him be told so; lest, perchance, i think We dare not [MASK] the question': The size of tensor a (515) must match the size of tensor b (512) at non-singleton dimension 1\n", + "Error processing chunk 'cheese, my digestion, why hast thou not brought thyself in to my house so many meals? Come, o Agamemnon? THERSITES. Thy commander, and Then tell me, Patroclus, about Achilles? PATROCLUS. Thy lord, and Then tell me, I ask thee, what’s Thersites? THERSITES. thy knower, Patroclus. Then tell , Patroclus, what art thou? and Thou must tell that , ACHILLES. O, tell, tell, or I’ll decline the whole . Agamemnon commands Achilles; Achilles , my lord; I am thy knower; and Patroclus is thy fool. PATROCLUS. You rascal! make Peace, fool! I have already done. ACHILLES. He is a privileg’d man. Proceed, Thersites. but Agamemnon is a fool; achilles is a fool; Thersites is a fool; and, as always Patroclus is a fool. i Derive this; come. THERSITES. achilles is a fool to attempt to command Achilles; Achilles is a fool to be afraid of Agamemnon; Thersites is a fool to serve such a fool; and this Patroclus is a fool positive. PATROCLUS. why am I a fool? i Make that demand of my Creator. It suffices me to art. Look you, who is here? Enter Agamemnon, Ulysses, and Diomedes, Ajax and Calchas. now Come, Patroclus, I’ll speak to nobody. Come in with the Thersites. [_Exit_.] THERSITES. Here is such patchery, such juggling, and such knavery. All the world is a whore and a cuckold—a good quarrel to make emulous factions and bleed to death upon. Now the great serpigo on the subject, and war and lechery confound . [_Exit_.] AGAMEMNON. Where is this PATROCLUS. Within his tent; his ill-dispos’d, my lord. AGAMEMNON. let it be known to all that we are here. thou shent our messengers; and we lay by Our appertainings, speak of him. Let him be told so; lest, perchance, i think We dare not [MASK] the question': The size of tensor a (515) must match the size of tensor b (512) at non-singleton dimension 1\n", + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.22201351933310776\n", + "T-Statistic: -3.1365710180077406\n", + "P-Value: 0.0518009301559042\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 168\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 169\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.23333333333333334\n", + "T-Statistic: -5.497267759562843\n", + "P-Value: 0.011845956731503078\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.26666666666666666\n", + "Average of Other Ratios: 0.21610169491525424\n", + "T-Statistic: -3.493722261155749\n", + "P-Value: 0.0396601427679115\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 169\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 170\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.2941256830601093\n", + "T-Statistic: -2.1752046582440823\n", + "P-Value: 0.1178783573581168\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1910546139359699\n", + "T-Statistic: -7.418137270026101\n", + "P-Value: 0.017691506692045566\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 170\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 171\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4098360655737705\n", + "Average of Other Ratios: 0.325\n", + "T-Statistic: -3.3934426229508206\n", + "P-Value: 0.04266659593484531\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.23192090395480225\n", + "T-Statistic: -3.3773352617852765\n", + "P-Value: 0.043176200293171145\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 171\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 172\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.18333333333333335\n", + "T-Statistic: -10.579553917424954\n", + "P-Value: 0.0018041539811479287\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.1432909604519774\n", + "T-Statistic: -1.8928833055825962\n", + "P-Value: 0.15471358909740393\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 172\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 173\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.20737704918032787\n", + "T-Statistic: -4.581868223019867\n", + "P-Value: 0.019519384720467742\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.1290018832391714\n", + "T-Statistic: -3.8230779561170367\n", + "P-Value: 0.06211218967841154\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 173\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 174\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.31489071038251365\n", + "T-Statistic: -3.002786519391192\n", + "P-Value: 0.05754103974524036\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.2193502824858757\n", + "T-Statistic: -4.75599598618577\n", + "P-Value: 0.017644508859181555\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 174\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 175\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.31666666666666665\n", + "Average of Other Ratios: 0.23674863387978146\n", + "T-Statistic: -3.2609722394321854\n", + "P-Value: 0.04709695971232935\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.1601694915254237\n", + "T-Statistic: -2.0803333919424123\n", + "P-Value: 0.12896148504661395\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 175\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 176\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.2693306010928962\n", + "T-Statistic: -5.216676982893284\n", + "P-Value: 0.013697022421412816\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.1901129943502825\n", + "T-Statistic: -3.182414988821109\n", + "P-Value: 0.05000120219940348\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 176\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 177\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.30703551912568305\n", + "T-Statistic: -26.318257342843076\n", + "P-Value: 0.00012035017922562646\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.2531779661016949\n", + "T-Statistic: -6.182185493474629\n", + "P-Value: 0.008522718517249426\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 177\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 178\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.27363387978142073\n", + "T-Statistic: -11.501316579943088\n", + "P-Value: 0.0014110204730729838\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.2658898305084746\n", + "T-Statistic: -4.202979499690264\n", + "P-Value: 0.024585862477064943\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 178\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 179\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.28750000000000003\n", + "T-Statistic: -6.210497654675213\n", + "P-Value: 0.00841340269771356\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.2192090395480226\n", + "T-Statistic: -3.4729797480883895\n", + "P-Value: 0.040259338911251656\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 179\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 180\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.20833333333333334\n", + "T-Statistic: -23.92854336374633\n", + "P-Value: 0.00015995510518182317\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.19399717514124296\n", + "T-Statistic: -7.840633887955942\n", + "P-Value: 0.004320681154632681\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 180\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 181\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36065573770491804\n", + "Average of Other Ratios: 0.3\n", + "T-Statistic: -2.5734050069412073\n", + "P-Value: 0.08224938731599696\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.19413841807909604\n", + "T-Statistic: -5.2860346002987235\n", + "P-Value: 0.013206166418066021\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 181\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 182\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.2833333333333333\n", + "T-Statistic: -3.914708631165495\n", + "P-Value: 0.029629359927274702\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.2078154425612053\n", + "T-Statistic: -2.134529747722321\n", + "P-Value: 0.16636576065135147\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 182\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 183\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.3\n", + "T-Statistic: -5.782581278907251\n", + "P-Value: 0.010285339228476416\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.22372881355932203\n", + "T-Statistic: -3.59486813709167\n", + "P-Value: 0.036895807895617604\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 183\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 184\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.2489754098360656\n", + "T-Statistic: -9.227144083318263\n", + "P-Value: 0.002692797704338326\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.12245762711864407\n", + "T-Statistic: -7.84890994931086\n", + "P-Value: 0.004307536410571643\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 184\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 185\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.275\n", + "T-Statistic: -4.488369320415114\n", + "P-Value: 0.02063340350086202\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2833333333333333\n", + "Average of Other Ratios: 0.21610169491525422\n", + "T-Statistic: -2.401102376173316\n", + "P-Value: 0.09577949808406833\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 185\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 186\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.2791666666666667\n", + "T-Statistic: -7.354405200045387\n", + "P-Value: 0.00519578998775744\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.20254237288135596\n", + "T-Statistic: -7.415534221028932\n", + "P-Value: 0.005073622594306211\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 186\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 187\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4098360655737705\n", + "Average of Other Ratios: 0.2708333333333333\n", + "T-Statistic: -9.766999910411144\n", + "P-Value: 0.002280530521151582\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.1772598870056497\n", + "T-Statistic: -6.333333333333335\n", + "P-Value: 0.007959883216421762\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 187\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 188\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3333333333333333\n", + "Average of Other Ratios: 0.22659380692167577\n", + "T-Statistic: -9.085817324099237\n", + "P-Value: 0.011897809590563803\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.18333333333333332\n", + "Average of Other Ratios: 0.15677966101694915\n", + "T-Statistic: -6.26666666666667\n", + "P-Value: 0.00820192086968827\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 188\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 189\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.25416666666666665\n", + "T-Statistic: -6.766299230685365\n", + "P-Value: 0.0065959876919741\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.19413841807909604\n", + "T-Statistic: -5.382088936904238\n", + "P-Value: 0.012563625893657768\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 189\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 190\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.27499999999999997\n", + "T-Statistic: -8.885189155139434\n", + "P-Value: 0.0030061660806281146\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.23545197740112994\n", + "T-Statistic: -1.9022556390977454\n", + "P-Value: 0.30811702486531156\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 190\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 191\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.34446721311475414\n", + "T-Statistic: -3.3256460363533833\n", + "P-Value: 0.04486462240497135\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.2754237288135593\n", + "T-Statistic: -9.191300234460838\n", + "P-Value: 0.0027235534322141448\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 191\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 192\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.26584699453551913\n", + "T-Statistic: -15.779833617982495\n", + "P-Value: 0.0005532481835581896\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.26666666666666666\n", + "Average of Other Ratios: 0.20338983050847456\n", + "T-Statistic: -2.536300556483895\n", + "P-Value: 0.08495405146875473\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 192\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 193\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.2067622950819672\n", + "T-Statistic: -6.043879310010205\n", + "P-Value: 0.009084400202355151\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1307909604519774\n", + "T-Statistic: -10.043517801177494\n", + "P-Value: 0.002101488368509634\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 193\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 194\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.225\n", + "T-Statistic: -9.957417838496047\n", + "P-Value: 0.0021551750735026\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.14766949152542375\n", + "T-Statistic: -6.856800905858473\n", + "P-Value: 0.006350573689746284\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 194\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 195\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.2583333333333333\n", + "T-Statistic: -3.3466253328190545\n", + "P-Value: 0.04416940727951125\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.16871468926553673\n", + "T-Statistic: -7.1482687865653824\n", + "P-Value: 0.005637501684561561\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 195\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 196\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.2692622950819672\n", + "T-Statistic: -4.2061085388157045\n", + "P-Value: 0.02453744768447861\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.1483050847457627\n", + "T-Statistic: -2.948006683383839\n", + "P-Value: 0.06012104778921462\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 196\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 197\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.23654371584699452\n", + "T-Statistic: -14.867978061029115\n", + "P-Value: 0.0006602177614744704\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.26666666666666666\n", + "Average of Other Ratios: 0.14830508474576273\n", + "T-Statistic: -4.1037036556074025\n", + "P-Value: 0.02618807381282589\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 197\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 198\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.1875\n", + "T-Statistic: -34.83606557377048\n", + "P-Value: 5.20111686902031e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.16885593220338985\n", + "T-Statistic: -6.912825719494809\n", + "P-Value: 0.0062046516135269335\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 198\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 199\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.20710382513661202\n", + "T-Statistic: -10.283901901516982\n", + "P-Value: 0.001960684970337362\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.21666666666666667\n", + "Average of Other Ratios: 0.1694915254237288\n", + "T-Statistic: -3.936227748605116\n", + "P-Value: 0.02920926328962092\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 199\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 200\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.28592896174863386\n", + "T-Statistic: -4.828799478606252\n", + "P-Value: 0.01693007546409189\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.19858757062146892\n", + "T-Statistic: -3.923159163483533\n", + "P-Value: 0.029463468310785026\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 200\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 201\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -7.851608784383327\n", + "P-Value: 0.004303261311903815\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -7.374780272477125\n", + "P-Value: 0.0051546431499948815\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 201\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 202\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -6.507337920439277\n", + "P-Value: 0.007370864736714179\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.346045197740113\n", + "T-Statistic: -6.302829818170098\n", + "P-Value: 0.008069446993235066\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 202\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 203\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.489275956284153\n", + "T-Statistic: -6.06668077104039\n", + "P-Value: 0.00898852350687363\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.38834745762711864\n", + "T-Statistic: -7.130307147698573\n", + "P-Value: 0.005678300191929151\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 203\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 204\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.373155737704918\n", + "T-Statistic: -5.4116120970679615\n", + "P-Value: 0.01237436958129694\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.36292372881355933\n", + "T-Statistic: -3.837369345431411\n", + "P-Value: 0.031204798309479358\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 204\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 205\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.3875\n", + "T-Statistic: -3.170655272045821\n", + "P-Value: 0.050455235210516786\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.37542372881355934\n", + "T-Statistic: -7.414573731136387\n", + "P-Value: 0.005075512686812505\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 205\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 206\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.4125\n", + "T-Statistic: -5.932056727240316\n", + "P-Value: 0.0095745359301512\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3686440677966102\n", + "T-Statistic: -4.46962233410428\n", + "P-Value: 0.020866618588713408\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 206\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 207\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.39999999999999997\n", + "T-Statistic: -4.741062246648036\n", + "P-Value: 0.017795875960800792\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3601694915254237\n", + "T-Statistic: -7.358286550031938\n", + "P-Value: 0.005187918406325382\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 207\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 208\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.39583333333333337\n", + "T-Statistic: -6.819438116767888\n", + "P-Value: 0.006450408520510395\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3504237288135593\n", + "T-Statistic: -6.6711622996447275\n", + "P-Value: 0.006867611409633449\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 208\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 209\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.41427595628415304\n", + "T-Statistic: -3.5454803247028934\n", + "P-Value: 0.03821378114440935\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3543785310734463\n", + "T-Statistic: -3.127498225142409\n", + "P-Value: 0.05216674452503871\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 209\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 210\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -10.982657455064276\n", + "P-Value: 0.0016163522474045274\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3331920903954802\n", + "T-Statistic: -1.7611959878594572\n", + "P-Value: 0.1764230443495778\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 210\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 211\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4810109289617486\n", + "T-Statistic: -2.397936955599478\n", + "P-Value: 0.09605254927181417\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4387005649717514\n", + "T-Statistic: -3.5958734600175015\n", + "P-Value: 0.03686958706248058\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 211\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 212\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.39166666666666666\n", + "T-Statistic: -4.380390416901878\n", + "P-Value: 0.022024716850259\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3257062146892655\n", + "T-Statistic: -6.924029541793238\n", + "P-Value: 0.02022773333106733\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 212\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 213\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.36249999999999993\n", + "T-Statistic: -5.412060981830556\n", + "P-Value: 0.012371520890864776\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3334745762711865\n", + "T-Statistic: -4.989086378646126\n", + "P-Value: 0.015484895041677766\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 213\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 214\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4185792349726776\n", + "T-Statistic: -3.1318929933163084\n", + "P-Value: 0.05198914240331166\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.29555084745762716\n", + "T-Statistic: -4.560758399398884\n", + "P-Value: 0.019763952369741263\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 214\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 215\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3980191256830601\n", + "T-Statistic: -6.2294612621620695\n", + "P-Value: 0.008341210430613992\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.37570621468926557\n", + "T-Statistic: -3.448652473575024\n", + "P-Value: 0.04097685603288608\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 215\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 216\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.43142076502732246\n", + "T-Statistic: -11.855239270966623\n", + "P-Value: 0.00129040506090979\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.388135593220339\n", + "T-Statistic: -4.65582342119235\n", + "P-Value: 0.018692942668843485\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 216\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 217\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4312158469945355\n", + "T-Statistic: -2.769971026007625\n", + "P-Value: 0.06957017387261798\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.37125706214689264\n", + "T-Statistic: -4.6417007257391365\n", + "P-Value: 0.018847202898111523\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 217\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 218\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.3818306010928962\n", + "T-Statistic: -4.847117449604144\n", + "P-Value: 0.016756266049925205\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3008474576271187\n", + "T-Statistic: -3.3764082011477075\n", + "P-Value: 0.04320576532803089\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 218\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 219\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.4064890710382514\n", + "T-Statistic: -1.6875505812729021\n", + "P-Value: 0.19008194720260654\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3799435028248587\n", + "T-Statistic: -2.299434047232845\n", + "P-Value: 0.10504286102353275\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 219\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 220\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -8.742170122442447\n", + "P-Value: 0.003151555065586373\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3674435028248587\n", + "T-Statistic: -3.575513903469711\n", + "P-Value: 0.03740527845126834\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 220\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 221\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.47274590163934427\n", + "T-Statistic: -3.8037548196566555\n", + "P-Value: 0.03192304772258143\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4132768361581921\n", + "T-Statistic: -2.677380542667445\n", + "P-Value: 0.07521629738306508\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 221\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 222\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.3689207650273224\n", + "T-Statistic: -6.200033744114341\n", + "P-Value: 0.00845358911672797\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3559322033898305\n", + "T-Statistic: -8.779860612843027\n", + "P-Value: 0.0031123529183814187\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 222\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 223\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.425\n", + "T-Statistic: -12.405173285892113\n", + "P-Value: 0.0011287421498101565\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.38210922787193974\n", + "T-Statistic: -2.7837850452128095\n", + "P-Value: 0.10845055325071953\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 223\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 224\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3605874316939891\n", + "T-Statistic: -5.648887116271631\n", + "P-Value: 0.010980796736842062\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.35812146892655367\n", + "T-Statistic: -2.5160267825812963\n", + "P-Value: 0.08647807452830744\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 224\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 225\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.39364754098360655\n", + "T-Statistic: -4.293991625472901\n", + "P-Value: 0.023226546554816672\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3601694915254237\n", + "T-Statistic: -3.233808333817773\n", + "P-Value: 0.0480764627944047\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 225\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 226\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.40614754098360656\n", + "T-Statistic: -4.422690436146169\n", + "P-Value: 0.02146562039496988\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.36694915254237287\n", + "T-Statistic: -2.474962294633946\n", + "P-Value: 0.08966910755373639\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 226\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 227\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.38148907103825136\n", + "T-Statistic: -8.30926126344939\n", + "P-Value: 0.0036525000991545153\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3294491525423729\n", + "T-Statistic: -5.302012165253797\n", + "P-Value: 0.013096358284056045\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 227\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 228\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.38749999999999996\n", + "T-Statistic: -3.4641024571780417\n", + "P-Value: 0.04051930159431608\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3333333333333333\n", + "T-Statistic: -4.82600482600724\n", + "P-Value: 0.016956798300543117\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 228\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 229\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3691939890710383\n", + "T-Statistic: -6.170366319105602\n", + "P-Value: 0.008568905214827621\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3177966101694915\n", + "T-Statistic: -7.026666666666668\n", + "P-Value: 0.0059214753537602605\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 229\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 230\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -3.05362313626129\n", + "P-Value: 0.05526992091860083\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.379590395480226\n", + "T-Statistic: -4.111518045317934\n", + "P-Value: 0.02605716031917639\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 230\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 231\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.42711748633879776\n", + "T-Statistic: -7.960605911691153\n", + "P-Value: 0.004135187577765976\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.41007532956685494\n", + "T-Statistic: -4.605203601134296\n", + "P-Value: 0.04405939035816047\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 231\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 232\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.45163934426229513\n", + "T-Statistic: -3.2153935344715583\n", + "P-Value: 0.04875516994578165\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.40127118644067794\n", + "T-Statistic: -2.4156666462852487\n", + "P-Value: 0.09453530050793599\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 232\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 233\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.43995901639344265\n", + "T-Statistic: -3.5143217545081513\n", + "P-Value: 0.03907627630046548\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3686440677966102\n", + "T-Statistic: -3.612819691689752\n", + "P-Value: 0.036431155449573864\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 233\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 234\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.3816256830601093\n", + "T-Statistic: -11.100144808896685\n", + "P-Value: 0.0015665315397856033\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.36299435028248583\n", + "T-Statistic: -4.890297438969469\n", + "P-Value: 0.016355645150575224\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 234\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 235\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4523224043715847\n", + "T-Statistic: -2.5642179367732165\n", + "P-Value: 0.082909074165789\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3626412429378531\n", + "T-Statistic: -2.3461567951496813\n", + "P-Value: 0.1006566092112587\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 235\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 236\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4187158469945355\n", + "T-Statistic: -3.611089663775107\n", + "P-Value: 0.03647560832552664\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.41035781544256117\n", + "T-Statistic: -3.6256912791125826\n", + "P-Value: 0.06836231728156876\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 236\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 237\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4362021857923497\n", + "T-Statistic: -2.402996731954332\n", + "P-Value: 0.09561654218317737\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6779661016949152\n", + "Average of Other Ratios: 0.413135593220339\n", + "T-Statistic: -6.782402329135958\n", + "P-Value: 0.006551418920014316\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 237\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 238\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.75\n", + "Average of Other Ratios: 0.44371584699453553\n", + "T-Statistic: -9.529761387107365\n", + "P-Value: 0.0024505949878125062\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5932203389830508\n", + "Average of Other Ratios: 0.40091807909604515\n", + "T-Statistic: -7.503655577513371\n", + "P-Value: 0.004904092057273042\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 238\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 239\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.42315573770491804\n", + "T-Statistic: -2.2737870570417353\n", + "P-Value: 0.10754893601572461\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3500706214689266\n", + "T-Statistic: -4.153796413823909\n", + "P-Value: 0.025363321264457474\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 239\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 240\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -7.346004713131698\n", + "P-Value: 0.005212880633794737\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.37645951035781544\n", + "T-Statistic: -9.627032312606932\n", + "P-Value: 0.010618294541056795\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 240\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 241\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -4.3698631465799025\n", + "P-Value: 0.022166781498566\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.307909604519774\n", + "T-Statistic: -2.524577979762879\n", + "P-Value: 0.08583120198777253\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 241\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 242\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -16.027311891144493\n", + "P-Value: 0.0005282450921087477\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.423728813559322\n", + "T-Statistic: -2.5287816912705736\n", + "P-Value: 0.08551538240691227\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 242\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 243\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.40642076502732244\n", + "T-Statistic: -3.4412842582299485\n", + "P-Value: 0.04119738288603227\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.32478813559322034\n", + "T-Statistic: -11.455677986899559\n", + "P-Value: 0.0014276482471677783\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 243\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 244\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.36666666666666664\n", + "T-Statistic: -3.8190928053365694\n", + "P-Value: 0.031592715134547304\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3050847457627119\n", + "T-Statistic: -5.140537735016466\n", + "P-Value: 0.01426366105655198\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 244\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 245\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.4458333333333333\n", + "T-Statistic: -2.433681073025334\n", + "P-Value: 0.09302352271601963\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3813559322033898\n", + "T-Statistic: -1.4070831677394464\n", + "P-Value: 0.25411009793550343\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 245\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 246\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.3774590163934426\n", + "T-Statistic: -8.635338782373333\n", + "P-Value: 0.00326628135311723\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.32083333333333336\n", + "T-Statistic: -3.313513183532759\n", + "P-Value: 0.045273022622265666\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 246\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 247\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.41448087431693986\n", + "T-Statistic: -2.7772482375001317\n", + "P-Value: 0.06914909220734049\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.35056497175141244\n", + "T-Statistic: -2.7353018723026903\n", + "P-Value: 0.07162056517888178\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 247\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 248\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.3916666666666667\n", + "T-Statistic: -2.4539091507908326\n", + "P-Value: 0.09136101012842844\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3629943502824859\n", + "T-Statistic: -5.564047150200574\n", + "P-Value: 0.011454300879796521\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 248\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 249\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.425\n", + "T-Statistic: -3.8366329971259767\n", + "P-Value: 0.031220308656298136\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3461158192090395\n", + "T-Statistic: -16.539568345323744\n", + "P-Value: 0.0004810744367490006\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 249\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 250\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.425\n", + "T-Statistic: -2.4919181614361894\n", + "P-Value: 0.08833429775357093\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3501412429378531\n", + "T-Statistic: -2.5736529074083476\n", + "P-Value: 0.08223167628896547\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 250\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 251\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4394808743169399\n", + "T-Statistic: -3.2347368286888813\n", + "P-Value: 0.048042559031082566\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.354590395480226\n", + "T-Statistic: -4.788344485301914\n", + "P-Value: 0.01732232163571842\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 251\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 252\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.41051912568306015\n", + "T-Statistic: -3.2435150269952304\n", + "P-Value: 0.04772351523132277\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3728813559322034\n", + "T-Statistic: -2.9121711386422233\n", + "P-Value: 0.061887533056045364\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 252\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 253\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.4291666666666667\n", + "T-Statistic: -3.4762904569519018\n", + "P-Value: 0.04016293189095533\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3759887005649718\n", + "T-Statistic: -2.087127965330441\n", + "P-Value: 0.17214627140969266\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 253\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 254\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4148224043715847\n", + "T-Statistic: -5.582808024011134\n", + "P-Value: 0.011347326397319939\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3898305084745763\n", + "T-Statistic: -5.7519673334545445\n", + "P-Value: 0.010439377744934933\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 254\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 255\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.39385245901639343\n", + "T-Statistic: -4.622249873571463\n", + "P-Value: 0.019062378557272608\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.35882768361581924\n", + "T-Statistic: -4.431514865270233\n", + "P-Value: 0.021351308327048055\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 255\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 256\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.42288251366120216\n", + "T-Statistic: -2.2047766760939767\n", + "P-Value: 0.11465918253571977\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.35169491525423735\n", + "T-Statistic: -2.667077795032928\n", + "P-Value: 0.07587912763738508\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 256\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 257\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.39392076502732243\n", + "T-Statistic: -3.234755633152603\n", + "P-Value: 0.048041872703208165\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.32627118644067793\n", + "T-Statistic: -2.8103535287436263\n", + "P-Value: 0.06727306836232257\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 257\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 258\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4519808743169399\n", + "T-Statistic: -1.7442611130916705\n", + "P-Value: 0.17946189402091586\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3484934086629002\n", + "T-Statistic: -4.258091646437124\n", + "P-Value: 0.0509728994156799\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 258\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 259\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.3791666666666667\n", + "T-Statistic: -4.961251862891206\n", + "P-Value: 0.01572399871994463\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3923022598870056\n", + "T-Statistic: -4.900769721140662\n", + "P-Value: 0.01626036695364969\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 259\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 260\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.3041666666666667\n", + "T-Statistic: -10.94515337164297\n", + "P-Value: 0.0016326978989827164\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.28700564971751413\n", + "T-Statistic: -6.458938312987853\n", + "P-Value: 0.007528780847255386\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 260\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 261\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.4235655737704918\n", + "T-Statistic: -2.1896107986499023\n", + "P-Value: 0.11629675807965059\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3671610169491525\n", + "T-Statistic: -3.7775875209558483\n", + "P-Value: 0.032496904050512185\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 261\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 262\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.43948087431693983\n", + "T-Statistic: -2.3754283562972716\n", + "P-Value: 0.09802174273100718\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.40084745762711865\n", + "T-Statistic: -5.117647058823531\n", + "P-Value: 0.014439978840531617\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 262\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 263\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.4208333333333333\n", + "T-Statistic: -4.533288093479356\n", + "P-Value: 0.020088194824742056\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.31631355932203387\n", + "T-Statistic: -4.390851056341401\n", + "P-Value: 0.021884716395917995\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 263\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 264\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3310109289617486\n", + "T-Statistic: -3.076310185278194\n", + "P-Value: 0.05429278214985614\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3333333333333333\n", + "Average of Other Ratios: 0.2711864406779661\n", + "T-Statistic: -4.016632088371216\n", + "P-Value: 0.02770576700479268\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 264\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 265\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.43586065573770494\n", + "T-Statistic: -3.3338420681873173\n", + "P-Value: 0.04459138016214428\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3540489642184557\n", + "T-Statistic: -3.260853543038344\n", + "P-Value: 0.08256523104344153\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 265\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 266\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4228825136612022\n", + "T-Statistic: -3.399472771732523\n", + "P-Value: 0.042477781423474185\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.37563559322033896\n", + "T-Statistic: -4.895597481406331\n", + "P-Value: 0.016307334114510047\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 266\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 267\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.4666666666666667\n", + "T-Statistic: -2.9838938061318583\n", + "P-Value: 0.0584148664790183\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.39682203389830506\n", + "T-Statistic: -4.774692154738101\n", + "P-Value: 0.017457356531658943\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 267\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 268\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.3833333333333333\n", + "T-Statistic: -8.868478725093105\n", + "P-Value: 0.003022689256232251\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3771186440677966\n", + "T-Statistic: -6.9688146846829495\n", + "P-Value: 0.00606320270460328\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 268\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 269\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.4437841530054645\n", + "T-Statistic: -11.678549121160296\n", + "P-Value: 0.0013488323998950805\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.40480225988700563\n", + "T-Statistic: -2.244619924618763\n", + "P-Value: 0.15392599376576382\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 269\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 270\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4271174863387978\n", + "T-Statistic: -3.7411614919740814\n", + "P-Value: 0.03331794416540563\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.33785310734463275\n", + "T-Statistic: -2.752891462679799\n", + "P-Value: 0.07057104088631745\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 270\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 271\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4189207650273224\n", + "T-Statistic: -4.8323202058438195\n", + "P-Value: 0.016896487730970963\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.32627118644067793\n", + "T-Statistic: -6.584419205304784\n", + "P-Value: 0.007128240934046268\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 271\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 272\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.344672131147541\n", + "T-Statistic: -3.571560837934814\n", + "P-Value: 0.03751043917785419\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.33340395480225987\n", + "T-Statistic: -2.392200090989989\n", + "P-Value: 0.09654983618738305\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 272\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 273\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3942622950819672\n", + "T-Statistic: -7.793998738706418\n", + "P-Value: 0.004395749962862252\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3500706214689266\n", + "T-Statistic: -2.147266074887937\n", + "P-Value: 0.12101972629670502\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 273\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 274\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.47677595628415304\n", + "T-Statistic: -2.363289165755069\n", + "P-Value: 0.09910414730873617\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.39896421845574387\n", + "T-Statistic: -4.392202321683557\n", + "P-Value: 0.048125248972546525\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 274\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 275\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.43586065573770494\n", + "T-Statistic: -4.152645529463587\n", + "P-Value: 0.025381891014694326\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.363135593220339\n", + "T-Statistic: -2.037441371070689\n", + "P-Value: 0.1343771965437019\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 275\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 276\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.40225409836065573\n", + "T-Statistic: -4.3808607027131465\n", + "P-Value: 0.022018397914196755\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3601694915254237\n", + "T-Statistic: -4.043803171028667\n", + "P-Value: 0.027220277010713455\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 276\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 277\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.41250000000000003\n", + "T-Statistic: -7.574148554850093\n", + "P-Value: 0.0047738189211758006\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.32916666666666666\n", + "T-Statistic: -4.334673395356122\n", + "P-Value: 0.0226503559908631\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 277\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 278\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.45833333333333337\n", + "T-Statistic: -2.975985583098297\n", + "P-Value: 0.058785578261159106\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.37711864406779666\n", + "T-Statistic: -1.9749677244040724\n", + "P-Value: 0.14275464845123928\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 278\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 279\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4522540983606557\n", + "T-Statistic: -3.550343434787011\n", + "P-Value: 0.03808136417851271\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4472457627118644\n", + "T-Statistic: -4.483251319120438\n", + "P-Value: 0.020696735526511162\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 279\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 280\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -3.9876185693143014\n", + "P-Value: 0.028236554956465203\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.346045197740113\n", + "T-Statistic: -5.143574545220902\n", + "P-Value: 0.014240481118726733\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 280\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 281\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -7.253159939163658\n", + "P-Value: 0.005406805253936257\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.29971751412429376\n", + "T-Statistic: -8.707841203347302\n", + "P-Value: 0.0031878312735672807\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 281\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 282\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.38749999999999996\n", + "T-Statistic: -3.8840535871493516\n", + "P-Value: 0.030241314676875894\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3206920903954802\n", + "T-Statistic: -6.159177470925122\n", + "P-Value: 0.008612932023026598\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 282\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 283\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7666666666666667\n", + "Average of Other Ratios: 0.36065573770491804\n", + "T-Statistic: -12.408708085543239\n", + "P-Value: 0.001127792567025593\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3248587570621469\n", + "T-Statistic: -5.276561879022925\n", + "P-Value: 0.01327183724912815\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 283\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 284\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.35000000000000003\n", + "T-Statistic: -6.992150468759068\n", + "P-Value: 0.006005502007535316\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.34597457627118644\n", + "T-Statistic: -4.470968814876207\n", + "P-Value: 0.020849754679967962\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 284\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 285\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4148907103825137\n", + "T-Statistic: -9.202675289866411\n", + "P-Value: 0.0027137429423814024\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.34590395480225994\n", + "T-Statistic: -3.714817178281528\n", + "P-Value: 0.033928358727363876\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 285\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 286\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.42288251366120216\n", + "T-Statistic: -4.288438027587506\n", + "P-Value: 0.023306668223673156\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3923728813559322\n", + "T-Statistic: -2.707175832648326\n", + "P-Value: 0.07333935451425552\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 286\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 287\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -2.327678697056396\n", + "P-Value: 0.10236424529812158\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3082627118644068\n", + "T-Statistic: -3.1270512783828903\n", + "P-Value: 0.052184849438917776\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 287\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 288\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.38333333333333336\n", + "T-Statistic: -8.1660987570467\n", + "P-Value: 0.0038411801830783828\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.37617702448210927\n", + "T-Statistic: -2.1147485270709563\n", + "P-Value: 0.16874539562177085\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 288\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 289\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.43196721311475417\n", + "T-Statistic: -3.728410507789727\n", + "P-Value: 0.03361162082090968\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.36292372881355933\n", + "T-Statistic: -5.215987959681293\n", + "P-Value: 0.013702017252772999\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 289\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 290\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.38333333333333336\n", + "T-Statistic: -6.688534635157998\n", + "P-Value: 0.0068169278980430605\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3816384180790961\n", + "T-Statistic: -2.042749961024093\n", + "P-Value: 0.17780819467640524\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 290\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 291\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.39385245901639343\n", + "T-Statistic: -3.766631201193383\n", + "P-Value: 0.03274110546757008\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3559322033898305\n", + "T-Statistic: -2.3848638865930605\n", + "P-Value: 0.09719033554345949\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 291\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 292\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3813524590163934\n", + "T-Statistic: -3.0906245783493898\n", + "P-Value: 0.05368742829459938\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3163841807909605\n", + "T-Statistic: -13.063945294843638\n", + "P-Value: 0.0009686387721898685\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 292\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 293\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -4.7424625152411375\n", + "P-Value: 0.01778161142649878\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.346045197740113\n", + "T-Statistic: -7.51860437612632\n", + "P-Value: 0.0048760779327184315\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 293\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 294\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.38162568306010936\n", + "T-Statistic: -6.039077465113362\n", + "P-Value: 0.009104762087317147\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.35\n", + "T-Statistic: -5.823074704906403\n", + "P-Value: 0.010086133108957392\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 294\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 295\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.5041666666666667\n", + "T-Statistic: -3.021862808739443\n", + "P-Value: 0.05667530663209718\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.4718455743879473\n", + "T-Statistic: -2.3238483653527293\n", + "P-Value: 0.145752005957584\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 295\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 296\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4146857923497268\n", + "T-Statistic: -3.8865083819708626\n", + "P-Value: 0.030191715950954285\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.38747645951035786\n", + "T-Statistic: -2.48387096774193\n", + "P-Value: 0.13098307174005927\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 296\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 297\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.476912568306011\n", + "T-Statistic: -3.077135925982908\n", + "P-Value: 0.0542576288739498\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3967514124293785\n", + "T-Statistic: -6.064766311184656\n", + "P-Value: 0.008996522307758008\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 297\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 298\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.39583333333333337\n", + "T-Statistic: -15.840015885207144\n", + "P-Value: 0.0005470254169839911\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3672316384180791\n", + "T-Statistic: -3.2003787654626485\n", + "P-Value: 0.04931755050578723\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 298\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 299\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7540983606557377\n", + "Average of Other Ratios: 0.39999999999999997\n", + "T-Statistic: -11.93916601780205\n", + "P-Value: 0.0012638276378236352\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3670197740112995\n", + "T-Statistic: -2.739338279816391\n", + "P-Value: 0.07137801893519374\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 299\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 300\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.40218579234972673\n", + "T-Statistic: -2.5598239902355586\n", + "P-Value: 0.08322688778357085\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3377118644067797\n", + "T-Statistic: -8.333345166769199\n", + "P-Value: 0.0036219707869269407\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 300\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 301\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.36666666666666664\n", + "T-Statistic: -11.112432660516188\n", + "P-Value: 0.0015614391657328315\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3432203389830508\n", + "T-Statistic: -3.466666666666669\n", + "P-Value: 0.04044399231953359\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 301\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 302\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.45416666666666666\n", + "T-Statistic: -2.293338139417192\n", + "P-Value: 0.10563206329335763\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3757062146892655\n", + "T-Statistic: -6.28618557093712\n", + "P-Value: 0.00813006680248645\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 302\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 303\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -2.937540923889767\n", + "P-Value: 0.06063034832622183\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3204802259887006\n", + "T-Statistic: -5.96246084588649\n", + "P-Value: 0.00943790429103949\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 303\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 304\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -7.838301347827194\n", + "P-Value: 0.004324395459725133\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3644067796610169\n", + "T-Statistic: -6.357755313912211\n", + "P-Value: 0.00787356840736677\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 304\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 305\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.39344262295081966\n", + "T-Statistic: -2.704003786107557\n", + "P-Value: 0.0735363944189725\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3771186440677966\n", + "T-Statistic: -1.5819079806991574\n", + "P-Value: 0.21181729689488749\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 305\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 306\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.4291666666666667\n", + "T-Statistic: -2.2939904432507365\n", + "P-Value: 0.10556882390727751\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4067796610169492\n", + "T-Statistic: -4.404557662822478\n", + "P-Value: 0.021703011848884367\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 306\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 307\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.43333333333333335\n", + "T-Statistic: -5.721871764168063\n", + "P-Value: 0.010593767203391514\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4101694915254237\n", + "T-Statistic: -2.577629100382969\n", + "P-Value: 0.12328440517539614\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 307\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 308\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4375\n", + "T-Statistic: -9.510009673164706\n", + "P-Value: 0.002465501797241344\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3753531073446328\n", + "T-Statistic: -6.31056255621062\n", + "P-Value: 0.008041485679774507\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 308\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 309\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4521174863387978\n", + "T-Statistic: -3.296192817472393\n", + "P-Value: 0.04586424145751324\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3672316384180791\n", + "T-Statistic: -7.5424723326565015\n", + "P-Value: 0.00483178537362738\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 309\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 310\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.373155737704918\n", + "T-Statistic: -3.7418090166847584\n", + "P-Value: 0.03330311842719912\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3145951035781544\n", + "T-Statistic: -3.1849674947750137\n", + "P-Value: 0.08604712307024512\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 310\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 311\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.39002732240437155\n", + "T-Statistic: -5.154995937878714\n", + "P-Value: 0.014153740439881685\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.32551789077212806\n", + "T-Statistic: -1.4349594508232855\n", + "P-Value: 0.2877633918978512\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 311\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 312\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.39767759562841526\n", + "T-Statistic: -2.5958139567349727\n", + "P-Value: 0.08066720661779468\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.354590395480226\n", + "T-Statistic: -3.4224483105094405\n", + "P-Value: 0.041768035740251584\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 312\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 313\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4650273224043716\n", + "T-Statistic: -3.1399952826589113\n", + "P-Value: 0.051663705233341434\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3714689265536723\n", + "T-Statistic: -5.34778311027372\n", + "P-Value: 0.012788321514374179\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 313\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 314\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3982923497267759\n", + "T-Statistic: -4.422992997264673\n", + "P-Value: 0.021461687900979676\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3771186440677966\n", + "T-Statistic: -2.881670261299946\n", + "P-Value: 0.06344247147107336\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 314\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 315\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.38934426229508196\n", + "T-Statistic: -3.7175780367875846\n", + "P-Value: 0.03386371975978312\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.33749999999999997\n", + "T-Statistic: -14.93808506039733\n", + "P-Value: 0.0006510638055588633\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 315\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 316\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.4191256830601093\n", + "T-Statistic: -2.474712698219785\n", + "P-Value: 0.08968894024591773\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3628531073446327\n", + "T-Statistic: -7.3695757821568195\n", + "P-Value: 0.005165112471402641\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 316\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 317\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.4208333333333334\n", + "T-Statistic: -6.523182985505984\n", + "P-Value: 0.007320113516282464\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.32937853107344633\n", + "T-Statistic: -2.297491967496297\n", + "P-Value: 0.10523014082412589\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 317\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 318\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.4375\n", + "T-Statistic: -5.669780163353537\n", + "P-Value: 0.010868137779207396\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.42627118644067796\n", + "T-Statistic: -4.07489509276445\n", + "P-Value: 0.02667808711215743\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 318\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 319\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.34795081967213115\n", + "T-Statistic: -4.836426389706701\n", + "P-Value: 0.01685742415885029\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3206920903954802\n", + "T-Statistic: -6.4099048762033455\n", + "P-Value: 0.00769333223799094\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 319\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 320\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.27499999999999997\n", + "T-Statistic: -6.209471659985009\n", + "P-Value: 0.008417331899586392\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.2196327683615819\n", + "T-Statistic: -3.2102314527592526\n", + "P-Value: 0.04894759775912512\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 320\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 321\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.4395491803278688\n", + "T-Statistic: -6.062960519560702\n", + "P-Value: 0.009004075645090898\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.3799435028248588\n", + "T-Statistic: -4.547561092589984\n", + "P-Value: 0.019918871326456516\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 321\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 322\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.38633879781420766\n", + "T-Statistic: -3.142740772481161\n", + "P-Value: 0.051554012753716585\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3415960451977401\n", + "T-Statistic: -2.7606608826884482\n", + "P-Value: 0.07011354688774671\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 322\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 323\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3895491803278689\n", + "T-Statistic: -2.647776278064909\n", + "P-Value: 0.07714046268331175\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.33778248587570625\n", + "T-Statistic: -2.679574844992895\n", + "P-Value: 0.07507605335291008\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 323\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 324\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.4375\n", + "T-Statistic: -3.5755004055064217\n", + "P-Value: 0.037405636887797994\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.36687853107344637\n", + "T-Statistic: -2.7295669647405267\n", + "P-Value: 0.07196693599133859\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 324\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 325\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.47288251366120215\n", + "T-Statistic: -1.789503264890946\n", + "P-Value: 0.17147429025836847\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3924435028248588\n", + "T-Statistic: -4.124951949583921\n", + "P-Value: 0.02583406935919925\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 325\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 326\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3976092896174863\n", + "T-Statistic: -2.616716369900444\n", + "P-Value: 0.07922508793306127\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3293079096045197\n", + "T-Statistic: -5.9823754058047705\n", + "P-Value: 0.009349794795095102\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 326\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 327\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.3458333333333334\n", + "T-Statistic: -12.446270789290725\n", + "P-Value: 0.0011177670196131112\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3120056497175141\n", + "T-Statistic: -4.032355280571854\n", + "P-Value: 0.027423480466249588\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 327\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 328\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4144125683060109\n", + "T-Statistic: -3.3818058014152355\n", + "P-Value: 0.04303398980095592\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.3836864406779661\n", + "T-Statistic: -4.836938005621383\n", + "P-Value: 0.01685256521081953\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 328\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 329\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.40225409836065573\n", + "T-Statistic: -1.9432287224055769\n", + "P-Value: 0.14724565777265242\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3209745762711864\n", + "T-Statistic: -4.989071878278333\n", + "P-Value: 0.015485018365118487\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 329\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 330\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.3625\n", + "T-Statistic: -6.568076588878649\n", + "P-Value: 0.007178794023354771\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3389830508474576\n", + "T-Statistic: -6.817746450746516\n", + "P-Value: 0.00645497746167015\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 330\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 331\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.38989071038251366\n", + "T-Statistic: -3.8433173783046315\n", + "P-Value: 0.031079871545519045\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3292372881355932\n", + "T-Statistic: -5.0198252557428855\n", + "P-Value: 0.015226316305783671\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 331\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 332\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.41058743169398904\n", + "T-Statistic: -5.412428171026809\n", + "P-Value: 0.012369191290195588\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3586158192090395\n", + "T-Statistic: -4.20590123701231\n", + "P-Value: 0.0245406514009237\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 332\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 333\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.4291666666666667\n", + "T-Statistic: -4.446685303445383\n", + "P-Value: 0.021156625841990514\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.36730225988700566\n", + "T-Statistic: -5.386626427441342\n", + "P-Value: 0.012534293739746833\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 333\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 334\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.41666666666666663\n", + "T-Statistic: -4.468875866263539\n", + "P-Value: 0.020875975290483854\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.385593220338983\n", + "T-Statistic: -9.000000000000007\n", + "P-Value: 0.002895812161864139\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 334\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 335\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4230191256830601\n", + "T-Statistic: -2.1008219346441472\n", + "P-Value: 0.12646621272270542\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3543785310734463\n", + "T-Statistic: -13.59402614992177\n", + "P-Value: 0.0008610534762829226\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 335\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 336\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.40710382513661203\n", + "T-Statistic: -4.176894082046699\n", + "P-Value: 0.02499431409045767\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.2635593220338983\n", + "T-Statistic: -2.8494467148061093\n", + "P-Value: 0.10425518121888491\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 336\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 337\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.41065573770491803\n", + "T-Statistic: -6.713645725875071\n", + "P-Value: 0.0067445330979899526\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3543785310734463\n", + "T-Statistic: -12.120686030907725\n", + "P-Value: 0.0012087821339881704\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 337\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 338\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.425\n", + "T-Statistic: -2.8842564548659144\n", + "P-Value: 0.06330874589287337\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.34745762711864403\n", + "T-Statistic: -9.372619697821948\n", + "P-Value: 0.00257258879538172\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 338\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 339\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.45416666666666666\n", + "T-Statistic: -6.130104668195616\n", + "P-Value: 0.008728726955771435\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.576271186440678\n", + "Average of Other Ratios: 0.3879943502824859\n", + "T-Statistic: -9.187860980455177\n", + "P-Value: 0.0027265288821396168\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 339\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 340\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -8.031429676406121\n", + "P-Value: 0.004030601648600872\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3543079096045198\n", + "T-Statistic: -3.2679435909119663\n", + "P-Value: 0.046849662447178565\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 340\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 341\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4480191256830601\n", + "T-Statistic: -3.7683602379617436\n", + "P-Value: 0.032702411681861464\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4048728813559322\n", + "T-Statistic: -6.267903613009531\n", + "P-Value: 0.008197342653735116\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 341\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 342\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.41885245901639345\n", + "T-Statistic: -1.6093008135111428\n", + "P-Value: 0.20592487686100341\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3246468926553672\n", + "T-Statistic: -2.4329184228681733\n", + "P-Value: 0.0930869236120761\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 342\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 343\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -6.267404269385084\n", + "P-Value: 0.008199190432380252\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.364406779661017\n", + "T-Statistic: -7.823426359338976\n", + "P-Value: 0.004348181653136324\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 343\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 344\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4438524590163934\n", + "T-Statistic: -5.090988202057138\n", + "P-Value: 0.014648921763255886\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.36313559322033895\n", + "T-Statistic: -3.4592703994453387\n", + "P-Value: 0.04066170197907394\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 344\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 345\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.3083333333333333\n", + "T-Statistic: -18.57762175574247\n", + "P-Value: 0.0003403993871209538\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.27429378531073445\n", + "T-Statistic: -13.77956011482232\n", + "P-Value: 0.0008271633740114473\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 345\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 346\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.40232240437158473\n", + "T-Statistic: -2.44235164169727\n", + "P-Value: 0.09230641363259132\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.34562146892655365\n", + "T-Statistic: -2.1945120664549007\n", + "P-Value: 0.11576447779308814\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 346\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 347\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.39801912568306014\n", + "T-Statistic: -2.3401785284145156\n", + "P-Value: 0.10120527554802543\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.37535310734463284\n", + "T-Statistic: -2.7799229360270217\n", + "P-Value: 0.06899512300862856\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 347\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 348\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3650273224043715\n", + "T-Statistic: -3.293760329942494\n", + "P-Value: 0.0459480546557983\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.28691148775894537\n", + "T-Statistic: -2.3579139690812547\n", + "P-Value: 0.14242127080654132\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 348\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 349\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.392167577413479\n", + "T-Statistic: -1.8864910288697916\n", + "P-Value: 0.19986672981680156\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.328954802259887\n", + "T-Statistic: -8.470193664445993\n", + "P-Value: 0.003454751941090446\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 349\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 350\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.45833333333333337\n", + "T-Statistic: -4.811368296463101\n", + "P-Value: 0.017097654477865952\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.4110169491525424\n", + "T-Statistic: -3.8450462874172215\n", + "P-Value: 0.031043679643556813\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 350\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 351\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3773224043715847\n", + "T-Statistic: -9.313966981164503\n", + "P-Value: 0.002620182057561568\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3502824858757062\n", + "T-Statistic: -8.315218406202995\n", + "P-Value: 0.003644917117727713\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 351\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 352\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4148224043715847\n", + "T-Statistic: -4.737661283448938\n", + "P-Value: 0.017830583546120875\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3632062146892655\n", + "T-Statistic: -2.5878167974480393\n", + "P-Value: 0.08122750520425058\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 352\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 353\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.4583333333333333\n", + "T-Statistic: -4.666486269355321\n", + "P-Value: 0.018577558529827218\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4110169491525424\n", + "T-Statistic: -9.907502030846539\n", + "P-Value: 0.0021871339690660045\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 353\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 354\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.43920765027322406\n", + "T-Statistic: -4.46624735000153\n", + "P-Value: 0.02090896608594079\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.417725988700565\n", + "T-Statistic: -3.7729601826121053\n", + "P-Value: 0.03259975536350789\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 354\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 355\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4187158469945355\n", + "T-Statistic: -1.9672355178709884\n", + "P-Value: 0.1438337407120422\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3757062146892655\n", + "T-Statistic: -6.25244138363679\n", + "P-Value: 0.00825481406798463\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 355\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 356\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -8.419686273573582\n", + "P-Value: 0.0035152556902358035\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3332627118644068\n", + "T-Statistic: -9.283339926398568\n", + "P-Value: 0.0026454980508478078\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 356\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 357\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.44166666666666665\n", + "T-Statistic: -3.447065049817408\n", + "P-Value: 0.041024239621887254\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.36320621468926556\n", + "T-Statistic: -4.276430704251948\n", + "P-Value: 0.023481122723639215\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 357\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 358\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.45\n", + "T-Statistic: -3.945284708663067\n", + "P-Value: 0.029034743151997156\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.40543785310734465\n", + "T-Statistic: -3.1308652437968307\n", + "P-Value: 0.05203060761024695\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 358\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 359\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.41045081967213115\n", + "T-Statistic: -3.0577746633809832\n", + "P-Value: 0.055089469506326795\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3670197740112995\n", + "T-Statistic: -3.3865448130535376\n", + "P-Value: 0.04288388807797634\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 359\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 360\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.4066256830601093\n", + "T-Statistic: -14.185315533396013\n", + "P-Value: 0.0007589934893947729\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3963983050847458\n", + "T-Statistic: -2.355804437045393\n", + "P-Value: 0.09977877653717733\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 360\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 361\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3400273224043716\n", + "T-Statistic: -5.098953338267703\n", + "P-Value: 0.0145860830962082\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.2953389830508475\n", + "T-Statistic: -13.691233125567791\n", + "P-Value: 0.000843071969469656\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 361\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 362\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4058743169398907\n", + "T-Statistic: -3.305417706655741\n", + "P-Value: 0.04554814619701029\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3983050847457627\n", + "T-Statistic: -3.3886778033477842\n", + "P-Value: 0.042816545584287505\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 362\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 363\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.4125\n", + "T-Statistic: -2.2397642698297733\n", + "P-Value: 0.11098584190703364\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3855932203389831\n", + "T-Statistic: -3.8368077899595163\n", + "P-Value: 0.03121662594701848\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 363\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 364\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -6.036355410889254\n", + "P-Value: 0.009116331366960937\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.30430790960451976\n", + "T-Statistic: -4.336743966663418\n", + "P-Value: 0.022621526682653798\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 364\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 365\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.4125\n", + "T-Statistic: -3.074072401423919\n", + "P-Value: 0.05438819286539707\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.35007062146892653\n", + "T-Statistic: -7.021230179838503\n", + "P-Value: 0.005934606781342949\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 365\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 366\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.3903005464480874\n", + "T-Statistic: -10.873299213730528\n", + "P-Value: 0.0016646323342411745\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.37521186440677967\n", + "T-Statistic: -3.216398138197251\n", + "P-Value: 0.04871783239364945\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 366\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 367\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.36249999999999993\n", + "T-Statistic: -8.995168397182145\n", + "P-Value: 0.0029003471322019392\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3418079096045198\n", + "T-Statistic: -3.841143835488625\n", + "P-Value: 0.031125448001670088\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 367\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 368\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.4208333333333333\n", + "T-Statistic: -6.6644480933382715\n", + "P-Value: 0.006887332838366486\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.34187853107344635\n", + "T-Statistic: -3.1832852004313463\n", + "P-Value: 0.04996780945878653\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 368\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 369\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.425\n", + "T-Statistic: -2.678526658699412\n", + "P-Value: 0.07514300545940161\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3686440677966102\n", + "T-Statistic: -5.6286160538198615\n", + "P-Value: 0.011091567639778482\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 369\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 370\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.43558743169398906\n", + "T-Statistic: -8.104167446872971\n", + "P-Value: 0.003926796249409396\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3332627118644068\n", + "T-Statistic: -6.9539103841401015\n", + "P-Value: 0.0061004380937106\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 370\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 371\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4191256830601093\n", + "T-Statistic: -4.775358960423008\n", + "P-Value: 0.017450729445301273\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.37584745762711863\n", + "T-Statistic: -2.9318808768994544\n", + "P-Value: 0.06090803764359439\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 371\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 372\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.40416666666666673\n", + "T-Statistic: -6.290438634313473\n", + "P-Value: 0.008114520022772869\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.4004237288135593\n", + "T-Statistic: -3.7578369991437737\n", + "P-Value: 0.03293881990782305\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 372\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 373\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7333333333333333\n", + "Average of Other Ratios: 0.3650956284153005\n", + "T-Statistic: -6.627801681197047\n", + "P-Value: 0.0069962968928984955\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.37966101694915255\n", + "T-Statistic: -5.662208585049302\n", + "P-Value: 0.010908789440745359\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 373\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 374\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.42295081967213116\n", + "T-Statistic: -2.3568974575090693\n", + "P-Value: 0.09967991090436344\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3548022598870057\n", + "T-Statistic: -3.46808249954851\n", + "P-Value: 0.04040248651626954\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 374\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 375\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.41045081967213115\n", + "T-Statistic: -4.3952411570175425\n", + "P-Value: 0.021826304453140925\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3711158192090396\n", + "T-Statistic: -2.4516068885038713\n", + "P-Value: 0.09154838744355702\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 375\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 376\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.41250000000000003\n", + "T-Statistic: -3.4877149221186383\n", + "P-Value: 0.03983250358959142\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.36271186440677966\n", + "T-Statistic: -9.86487655643415\n", + "P-Value: 0.0022149241738372996\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 376\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 377\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4026639344262295\n", + "T-Statistic: -4.885900809261954\n", + "P-Value: 0.016395862833158793\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3713983050847458\n", + "T-Statistic: -8.050860563356986\n", + "P-Value: 0.0040025200155213245\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 377\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 378\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -7.6296187670353355\n", + "P-Value: 0.004674501764841112\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.32507062146892657\n", + "T-Statistic: -2.8491875970653315\n", + "P-Value: 0.06515249712984865\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 378\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 379\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.4458333333333333\n", + "T-Statistic: -6.638664993068123\n", + "P-Value: 0.006963760953222826\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.36306497175141245\n", + "T-Statistic: -6.0781754491615825\n", + "P-Value: 0.008940693333973865\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 379\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 380\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.41912568306010933\n", + "T-Statistic: -17.09897132088851\n", + "P-Value: 0.00043575164889100897\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4406779661016949\n", + "T-Statistic: -5.490939506738963\n", + "P-Value: 0.011883974620146622\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 380\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 381\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.44829234972677595\n", + "T-Statistic: -5.766710653623844\n", + "P-Value: 0.010364820031007032\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.4046610169491526\n", + "T-Statistic: -3.190034610277395\n", + "P-Value: 0.04970976762287175\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 381\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 382\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.4125\n", + "T-Statistic: -5.5278108432930315\n", + "P-Value: 0.01166468471503337\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.33375706214689266\n", + "T-Statistic: -2.933998334563753\n", + "P-Value: 0.06080396633288608\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 382\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 383\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.425\n", + "T-Statistic: -10.350486199201907\n", + "P-Value: 0.0019239067953852225\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3626412429378531\n", + "T-Statistic: -2.0904012105524146\n", + "P-Value: 0.1277280900520854\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 383\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 384\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4314890710382514\n", + "T-Statistic: -3.272785517027942\n", + "P-Value: 0.046678870059437404\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3504237288135593\n", + "T-Statistic: -5.129835675678766\n", + "P-Value: 0.014345743161730394\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 384\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 385\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.39583333333333337\n", + "T-Statistic: -10.70159618814971\n", + "P-Value: 0.0017443718008383813\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3543079096045198\n", + "T-Statistic: -4.181753231671517\n", + "P-Value: 0.02491756822152362\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 385\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 386\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -4.125557008441323\n", + "P-Value: 0.025824079480551702\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.346045197740113\n", + "T-Statistic: -2.667891875399661\n", + "P-Value: 0.07582649117357966\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 386\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 387\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.39385245901639343\n", + "T-Statistic: -8.973702530064598\n", + "P-Value: 0.0029206091675186585\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3714689265536723\n", + "T-Statistic: -5.879572942861632\n", + "P-Value: 0.009816556345478428\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 387\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 388\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.41885245901639345\n", + "T-Statistic: -12.364286659593969\n", + "P-Value: 0.0011398034481367935\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.37966101694915255\n", + "T-Statistic: -6.598309513974847\n", + "P-Value: 0.007085641181587867\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 388\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 389\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -2.801466195764125\n", + "P-Value: 0.06777041139462144\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3627824858757063\n", + "T-Statistic: -4.484419263456089\n", + "P-Value: 0.020682260794596567\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 389\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 390\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.4166666666666667\n", + "T-Statistic: -7.828769077961604\n", + "P-Value: 0.004339618418212919\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.3836864406779661\n", + "T-Statistic: -4.199186687635008\n", + "P-Value: 0.024644713133568197\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 390\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 391\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.3458333333333333\n", + "T-Statistic: -12.230197713801166\n", + "P-Value: 0.0011771054889125297\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.35444915254237286\n", + "T-Statistic: -7.140350877192979\n", + "P-Value: 0.0056554388389449695\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 391\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 392\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -4.3462339347627506\n", + "P-Value: 0.02249000005004558\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3755649717514124\n", + "T-Statistic: -4.427578329769953\n", + "P-Value: 0.021402204751343277\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 392\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 393\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.43333333333333335\n", + "T-Statistic: -9.683827326548414\n", + "P-Value: 0.0023383191790880323\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.37288135593220345\n", + "T-Statistic: -2.6341579069577183\n", + "P-Value: 0.0780460308657842\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 393\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 394\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.4439207650273224\n", + "T-Statistic: -2.1366037206586563\n", + "P-Value: 0.12224492481984166\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.33719397363465164\n", + "T-Statistic: -2.622926709469525\n", + "P-Value: 0.1197907664196296\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 394\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 395\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.41666666666666663\n", + "T-Statistic: -23.25576104817017\n", + "P-Value: 0.00017417898616924437\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.36299435028248583\n", + "T-Statistic: -8.553684424935213\n", + "P-Value: 0.003357721498987871\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 395\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 396\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3983606557377049\n", + "T-Statistic: -14.711044641258004\n", + "P-Value: 0.0006813380311103391\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.33326271186440676\n", + "T-Statistic: -18.123845744121766\n", + "P-Value: 0.00036642093322613114\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 396\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 397\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.44166666666666665\n", + "T-Statistic: -3.8282796790862137\n", + "P-Value: 0.0313969565110335\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.36864406779661013\n", + "T-Statistic: -3.142183764359604\n", + "P-Value: 0.051576243518155256\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 397\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 398\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3980874316939891\n", + "T-Statistic: -5.915897985549872\n", + "P-Value: 0.009648207488286976\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3389830508474576\n", + "T-Statistic: -7.756717518813398\n", + "P-Value: 0.004457004963920533\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 398\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 399\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4291666666666667\n", + "T-Statistic: -4.396123870194077\n", + "P-Value: 0.021814583998735686\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3290960451977401\n", + "T-Statistic: -5.1956627047447785\n", + "P-Value: 0.013850433541822478\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 399\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 400\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -27.40047589022894\n", + "P-Value: 0.00010668857406321375\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3206920903954802\n", + "T-Statistic: -4.95328158335475\n", + "P-Value: 0.015793349475275394\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 400\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 401\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.4416666666666667\n", + "T-Statistic: -5.458841849494081\n", + "P-Value: 0.012079282996343666\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4135593220338983\n", + "T-Statistic: -3.3916011823151724\n", + "P-Value: 0.04272446714466854\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 401\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 402\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.39166666666666666\n", + "T-Statistic: -7.569913857342007\n", + "P-Value: 0.004781514945095522\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3248587570621469\n", + "T-Statistic: -6.203180801394921\n", + "P-Value: 0.008441476456348773\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 402\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 403\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.44781420765027324\n", + "T-Statistic: -3.9576539242406734\n", + "P-Value: 0.02879855831206138\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.40077683615819215\n", + "T-Statistic: -5.011757995152538\n", + "P-Value: 0.015293631034164368\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 403\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 404\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.43333333333333335\n", + "T-Statistic: -3.3050391298697224\n", + "P-Value: 0.04556106387896335\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.39237288135593223\n", + "T-Statistic: -11.151144229232496\n", + "P-Value: 0.0015455390866235931\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 404\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 405\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.275\n", + "T-Statistic: -20.058020052863462\n", + "P-Value: 0.0002708532723360041\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.28714689265536725\n", + "T-Statistic: -3.409092533983689\n", + "P-Value: 0.04217876250784662\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 405\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 406\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.38196721311475407\n", + "T-Statistic: -10.445379856844973\n", + "P-Value: 0.001873056150700024\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.33173258003766476\n", + "T-Statistic: -3.524320965413247\n", + "P-Value: 0.07193114099352556\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 406\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 407\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.35655737704918034\n", + "T-Statistic: -5.1034502548832\n", + "P-Value: 0.01455076115150943\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3305084745762712\n", + "T-Statistic: -2.3067562719090997\n", + "P-Value: 0.10434037801243047\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 407\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 408\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8032786885245902\n", + "Average of Other Ratios: 0.3791666666666667\n", + "T-Statistic: -15.40340748070682\n", + "P-Value: 0.0005943876559434028\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.36016949152542377\n", + "T-Statistic: -2.4527621833439794\n", + "P-Value: 0.09145430123164801\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 408\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 409\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -4.047798691785726\n", + "P-Value: 0.02714981201550245\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3483050847457627\n", + "T-Statistic: -11.775818566563489\n", + "P-Value: 0.00713429023847155\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 409\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 410\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.44398907103825136\n", + "T-Statistic: -3.5630178449635013\n", + "P-Value: 0.0377389945733\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.38806497175141247\n", + "T-Statistic: -2.8670514268084752\n", + "P-Value: 0.06420504585478538\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 410\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 411\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.4666666666666667\n", + "T-Statistic: -6.095443431544011\n", + "P-Value: 0.008869465900928337\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.4092514124293785\n", + "T-Statistic: -2.568138862870189\n", + "P-Value: 0.08262673431862806\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 411\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 412\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4312841530054645\n", + "T-Statistic: -3.2330608226780675\n", + "P-Value: 0.0481037799750477\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3839689265536724\n", + "T-Statistic: -2.903867111427604\n", + "P-Value: 0.06230610742939453\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 412\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 413\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.4416666666666667\n", + "T-Statistic: -2.8905475311707214\n", + "P-Value: 0.06298491773711792\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3940677966101695\n", + "T-Statistic: -6.80809631531433\n", + "P-Value: 0.006481122703408014\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 413\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 414\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4110655737704918\n", + "T-Statistic: -5.966922212523975\n", + "P-Value: 0.009418071176895705\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3542372881355932\n", + "T-Statistic: -2.2861298780156964\n", + "P-Value: 0.10633394781548948\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 414\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 415\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.75\n", + "Average of Other Ratios: 0.36495901639344264\n", + "T-Statistic: -15.214865853342191\n", + "P-Value: 0.0006165289089404656\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.38425141242937855\n", + "T-Statistic: -2.687063737163629\n", + "P-Value: 0.07459984908288483\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 415\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 416\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.3916666666666667\n", + "T-Statistic: -33.655737704917975\n", + "P-Value: 5.766533760280624e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3536723163841808\n", + "T-Statistic: -3.172729727206767\n", + "P-Value: 0.08662826593910922\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 416\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 417\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4230874316939891\n", + "T-Statistic: -3.063975004864087\n", + "P-Value: 0.054821341360087615\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.35593220338983045\n", + "T-Statistic: -4.843649660017283\n", + "P-Value: 0.016788991301625738\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 417\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 418\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.42500000000000004\n", + "T-Statistic: -7.268455569166599\n", + "P-Value: 0.005374210400514755\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3670197740112994\n", + "T-Statistic: -2.9579994448141673\n", + "P-Value: 0.05963975694231845\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 418\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 419\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -7.4933789577565\n", + "P-Value: 0.004923473774846551\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.30903954802259886\n", + "T-Statistic: -2.5908795387845776\n", + "P-Value: 0.12224781736312054\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 419\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 420\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.48750000000000004\n", + "T-Statistic: -4.140983606557374\n", + "P-Value: 0.025571051486798255\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.576271186440678\n", + "Average of Other Ratios: 0.4008474576271186\n", + "T-Statistic: -8.297720726530276\n", + "P-Value: 0.0036672497469535698\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 420\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 421\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.45416666666666666\n", + "T-Statistic: -5.024423429015315\n", + "P-Value: 0.015188120980305192\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.38182674199623357\n", + "T-Statistic: -1.9595943025088522\n", + "P-Value: 0.1891147904640091\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 421\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 422\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.34426229508196726\n", + "T-Statistic: -10.200056186904932\n", + "P-Value: 0.0020083355064274685\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.32076271186440675\n", + "T-Statistic: -4.322021374340152\n", + "P-Value: 0.022827550876124904\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 422\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 423\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.75\n", + "Average of Other Ratios: 0.4724043715846995\n", + "T-Statistic: -4.432281354966694\n", + "P-Value: 0.021341416432768157\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4388418079096046\n", + "T-Statistic: -2.947245022552593\n", + "P-Value: 0.0601579316359931\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 423\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 424\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.435724043715847\n", + "T-Statistic: -7.812794009945692\n", + "P-Value: 0.004365289535356393\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.39632768361581916\n", + "T-Statistic: -4.060240963855425\n", + "P-Value: 0.02693187975555432\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 424\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 425\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -5.133817644374452\n", + "P-Value: 0.014315130666193885\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3418079096045198\n", + "T-Statistic: -3.27165152540788\n", + "P-Value: 0.04671879927774851\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 425\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 426\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.42083333333333334\n", + "T-Statistic: -5.4872114695067635\n", + "P-Value: 0.011906446056017608\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3483992467043315\n", + "T-Statistic: -5.499999999999996\n", + "P-Value: 0.031504003041813854\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 426\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 427\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.44583333333333336\n", + "T-Statistic: -5.613383822262529\n", + "P-Value: 0.011175768196393236\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4279661016949152\n", + "T-Statistic: -10.734900802433872\n", + "P-Value: 0.0017285150870629262\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 427\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 428\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.4666666666666667\n", + "T-Statistic: -3.3346995013274205\n", + "P-Value: 0.044562916865554185\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.34230225988700563\n", + "T-Statistic: -3.432024351211093\n", + "P-Value: 0.041476672849086224\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 428\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 429\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3775273224043716\n", + "T-Statistic: -6.826452300266374\n", + "P-Value: 0.006431509604653321\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.35021186440677965\n", + "T-Statistic: -10.964467451419276\n", + "P-Value: 0.0016242528673193293\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 429\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 430\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3818989071038251\n", + "T-Statistic: -7.577650824830856\n", + "P-Value: 0.004767466337140692\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.37577683615819213\n", + "T-Statistic: -5.036096342175906\n", + "P-Value: 0.015091717648582546\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 430\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 431\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4771857923497268\n", + "T-Statistic: -2.244146678485596\n", + "P-Value: 0.11053578481406542\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.4134887005649718\n", + "T-Statistic: -4.473097699753992\n", + "P-Value: 0.020823127617620334\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 431\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 432\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -3.3233585459170785\n", + "P-Value: 0.044941262424214\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.3081214689265537\n", + "T-Statistic: -3.5637008822210965\n", + "P-Value: 0.037720655653052076\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 432\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 433\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.4125000000000001\n", + "T-Statistic: -6.066153107963813\n", + "P-Value: 0.008990727202934024\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.37563559322033896\n", + "T-Statistic: -4.420794483496409\n", + "P-Value: 0.02149028401764827\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 433\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 434\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -29.50232973023399\n", + "P-Value: 8.552802430562091e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.34745762711864403\n", + "T-Statistic: -5.845641436961716\n", + "P-Value: 0.009977307916285254\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 434\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 435\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.35628415300546445\n", + "T-Statistic: -6.079870046364211\n", + "P-Value: 0.008933670268337511\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3375706214689265\n", + "T-Statistic: -16.999999999999993\n", + "P-Value: 0.00044334353831207803\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 435\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 436\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7333333333333333\n", + "Average of Other Ratios: 0.42356557377049187\n", + "T-Statistic: -9.685668717829044\n", + "P-Value: 0.002337018895349377\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.4263418079096045\n", + "T-Statistic: -4.09478757977338\n", + "P-Value: 0.026338481224541917\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 436\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 437\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.38333333333333336\n", + "T-Statistic: -11.898312771791307\n", + "P-Value: 0.001276673082635795\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3378531073446328\n", + "T-Statistic: -3.7042710296105374\n", + "P-Value: 0.034176730688008236\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 437\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 438\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.45642076502732243\n", + "T-Statistic: -6.157320526109763\n", + "P-Value: 0.008620267599043325\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.39661016949152544\n", + "T-Statistic: -3.8969992136018448\n", + "P-Value: 0.029980924038161237\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 438\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 439\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.35273224043715845\n", + "T-Statistic: -11.787092032898832\n", + "P-Value: 0.0013125340009585288\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3305084745762712\n", + "T-Statistic: -3.6406043454349715\n", + "P-Value: 0.03572662155623015\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 439\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 440\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.48750000000000004\n", + "T-Statistic: -4.044451296869408\n", + "P-Value: 0.027208830672879287\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.42203389830508475\n", + "T-Statistic: -6.715954603984051\n", + "P-Value: 0.006737927535009723\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 440\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 441\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.32916666666666666\n", + "T-Statistic: -8.618610742527247\n", + "P-Value: 0.003284743377152693\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.32916666666666666\n", + "T-Statistic: -13.44549998474748\n", + "P-Value: 0.0008895256433019566\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 441\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 442\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8360655737704918\n", + "Average of Other Ratios: 0.3625\n", + "T-Statistic: -21.607919158456742\n", + "P-Value: 0.00021691691845571816\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3208333333333333\n", + "T-Statistic: -3.6177990004181297\n", + "P-Value: 0.0363035983070319\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 442\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 443\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.425\n", + "T-Statistic: -12.578664061524803\n", + "P-Value: 0.0010833606619678423\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.37288135593220345\n", + "T-Statistic: -2.5220141862320173\n", + "P-Value: 0.08602451986599671\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 443\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 444\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.4166666666666667\n", + "T-Statistic: -5.772363123361995\n", + "P-Value: 0.010336420243029285\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3940677966101695\n", + "T-Statistic: -5.016126374955461\n", + "P-Value: 0.01525713242925573\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 444\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 445\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.43968579234972677\n", + "T-Statistic: -2.8833841562530913\n", + "P-Value: 0.06335381086618001\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.39223163841807906\n", + "T-Statistic: -3.5593682026592988\n", + "P-Value: 0.03783717712998915\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 445\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 446\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4685109289617486\n", + "T-Statistic: -3.74670122389729\n", + "P-Value: 0.03319137901399563\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.41744350282485876\n", + "T-Statistic: -3.7183243247458044\n", + "P-Value: 0.03384627426695061\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 446\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 447\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.3916666666666667\n", + "T-Statistic: -4.201654069631227\n", + "P-Value: 0.02460640768982449\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.35593220338983056\n", + "T-Statistic: -4.164132562731401\n", + "P-Value: 0.025197328469594847\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 447\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 448\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4519808743169399\n", + "T-Statistic: -3.088509743881098\n", + "P-Value: 0.05377632746313144\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.37584745762711863\n", + "T-Statistic: -5.567318968997769\n", + "P-Value: 0.01143555025754969\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 448\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 449\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.464412568306011\n", + "T-Statistic: -3.9929846359991004\n", + "P-Value: 0.0281374053103564\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.43036723163841806\n", + "T-Statistic: -3.6908520053100666\n", + "P-Value: 0.03449613636026554\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 449\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 450\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8688524590163934\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -111.52459016393443\n", + "P-Value: 1.5893997832869387e-06\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.4135593220338983\n", + "T-Statistic: -6.287241983947008\n", + "P-Value: 0.008126201507413536\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 450\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 451\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.3625\n", + "T-Statistic: -10.053864168618263\n", + "P-Value: 0.002095156783314077\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.354590395480226\n", + "T-Statistic: -4.195847013586463\n", + "P-Value: 0.024696683385781026\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 451\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 452\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.46249999999999997\n", + "T-Statistic: -4.30939544513269\n", + "P-Value: 0.023006171549745307\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.3545197740112994\n", + "T-Statistic: -7.100680549678681\n", + "P-Value: 0.0057464492134181155\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 452\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 453\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.41878415300546445\n", + "T-Statistic: -3.907188916118307\n", + "P-Value: 0.029777989336963344\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3925141242937853\n", + "T-Statistic: -2.652993667727985\n", + "P-Value: 0.07679697042443225\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 453\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 454\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4311475409836066\n", + "T-Statistic: -2.3185802321951905\n", + "P-Value: 0.10321797628133385\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.32492937853107345\n", + "T-Statistic: -10.457966297708378\n", + "P-Value: 0.0018664458731184775\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 454\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 455\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4395491803278689\n", + "T-Statistic: -3.7611226340766057\n", + "P-Value: 0.03286477251791561\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.35868644067796607\n", + "T-Statistic: -13.182726027622682\n", + "P-Value: 0.000943041742019242\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 455\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 456\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.42500000000000004\n", + "T-Statistic: -7.6310252969080015\n", + "P-Value: 0.004672019040052037\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3855932203389831\n", + "T-Statistic: -4.476919366879844\n", + "P-Value: 0.0207754384968935\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 456\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 457\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3772540983606557\n", + "T-Statistic: -2.892862105157335\n", + "P-Value: 0.06286629800985993\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.3038135593220339\n", + "T-Statistic: -2.8400584700013614\n", + "P-Value: 0.06564341941663877\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 457\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 458\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3484972677595628\n", + "T-Statistic: -4.079918215864471\n", + "P-Value: 0.026591801968465816\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3038841807909604\n", + "T-Statistic: -4.6304395053630865\n", + "P-Value: 0.01897139383687084\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 458\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 459\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.35689890710382516\n", + "T-Statistic: -3.012713968246357\n", + "P-Value: 0.057088449146294205\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.29950564971751414\n", + "T-Statistic: -3.9160409861832215\n", + "P-Value: 0.02960312502298523\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 459\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 460\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4519808743169399\n", + "T-Statistic: -3.585650713444847\n", + "P-Value: 0.03713733022490143\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.4095338983050847\n", + "T-Statistic: -3.343711500585689\n", + "P-Value: 0.04426514489947514\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 460\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 461\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.39829234972677596\n", + "T-Statistic: -6.228521035735816\n", + "P-Value: 0.008344770553796356\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3516949152542373\n", + "T-Statistic: -2.764906125414197\n", + "P-Value: 0.06986512747198882\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 461\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 462\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.36495901639344264\n", + "T-Statistic: -11.467033890261218\n", + "P-Value: 0.001423486681886134\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3331920903954802\n", + "T-Statistic: -3.755601850914824\n", + "P-Value: 0.03298931499378954\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 462\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 463\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -18.492347410557112\n", + "P-Value: 0.00034509721387531697\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3418079096045198\n", + "T-Statistic: -5.430582663966678\n", + "P-Value: 0.012254723173754265\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 463\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 464\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -7.031379503531193\n", + "P-Value: 0.005910122809393594\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.342090395480226\n", + "T-Statistic: -2.3719428909642666\n", + "P-Value: 0.09833105019693004\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 464\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 465\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.4273224043715847\n", + "T-Statistic: -7.050049697276166\n", + "P-Value: 0.005865429795204114\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.3714689265536723\n", + "T-Statistic: -9.312498469112693\n", + "P-Value: 0.002621388573795643\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 465\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 466\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.36475409836065575\n", + "T-Statistic: -3.2441722541392934\n", + "P-Value: 0.047699736310779436\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3089453860640301\n", + "T-Statistic: -4.904381946588596\n", + "P-Value: 0.03914970069711023\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 466\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 467\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.40416666666666673\n", + "T-Statistic: -4.882283495355984\n", + "P-Value: 0.016429048371143395\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.40120056497175144\n", + "T-Statistic: -3.3115695002057812\n", + "P-Value: 0.04533888578886271\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 467\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 468\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.40286885245901644\n", + "T-Statistic: -2.273916596472962\n", + "P-Value: 0.10753609772746466\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.34611581920903955\n", + "T-Statistic: -5.009915149758147\n", + "P-Value: 0.015309062462291299\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 468\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 469\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.43176229508196723\n", + "T-Statistic: -4.998106522840042\n", + "P-Value: 0.01540842738464649\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3670197740112994\n", + "T-Statistic: -9.048660230550595\n", + "P-Value: 0.002850658340600586\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 469\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 470\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.44036885245901636\n", + "T-Statistic: -2.6121434779788526\n", + "P-Value: 0.07953784934392538\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.34138418079096045\n", + "T-Statistic: -2.360589363770645\n", + "P-Value: 0.09934685042336319\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 470\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 471\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.48524590163934433\n", + "T-Statistic: -1.929654958496209\n", + "P-Value: 0.1492167828881105\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3801553672316384\n", + "T-Statistic: -2.8398703872208553\n", + "P-Value: 0.06565358206650312\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 471\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 472\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4275273224043716\n", + "T-Statistic: -3.5561117736258945\n", + "P-Value: 0.037925056755999366\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3372175141242938\n", + "T-Statistic: -3.5534330922688557\n", + "P-Value: 0.037997540271468185\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 472\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 473\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.4625\n", + "T-Statistic: -4.040627784984222\n", + "P-Value: 0.027276446171401562\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.44491525423728817\n", + "T-Statistic: -2.26300952742407\n", + "P-Value: 0.10862358012074654\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 473\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 474\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.475\n", + "T-Statistic: -6.831662693460199\n", + "P-Value: 0.006417517963111248\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.41779661016949154\n", + "T-Statistic: -4.370123877984895\n", + "P-Value: 0.02216324865119666\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 474\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 475\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -24.012092343181642\n", + "P-Value: 0.0001582981185228392\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.32033898305084746\n", + "T-Statistic: -9.161458990088187\n", + "P-Value: 0.011705573839571451\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 475\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 476\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -8.579921461021282\n", + "P-Value: 0.003327975019105188\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.384180790960452\n", + "T-Statistic: -3.706246583305507\n", + "P-Value: 0.03413002778125795\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 476\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 477\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.3778688524590164\n", + "T-Statistic: -5.101866069096539\n", + "P-Value: 0.014563191721761432\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3247881355932203\n", + "T-Statistic: -4.8167110843371415\n", + "P-Value: 0.017046061853773893\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 477\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 478\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.385724043715847\n", + "T-Statistic: -7.664492373372008\n", + "P-Value: 0.004613454840878532\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.35444915254237286\n", + "T-Statistic: -3.7842912965837407\n", + "P-Value: 0.032348636596183485\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 478\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 479\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.4031876138433515\n", + "T-Statistic: -3.0047783082409976\n", + "P-Value: 0.09520452944787951\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.32492937853107345\n", + "T-Statistic: -3.4140132990801026\n", + "P-Value: 0.04202684058034692\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 479\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 480\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.4375\n", + "T-Statistic: -5.252662296910844\n", + "P-Value: 0.013439429896053543\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.3884180790960452\n", + "T-Statistic: -7.8459916831607694\n", + "P-Value: 0.004312165413432141\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 480\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 481\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.35280054644808745\n", + "T-Statistic: -2.803386864259781\n", + "P-Value: 0.06766254223000473\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.2825564971751412\n", + "T-Statistic: -8.059631433861888\n", + "P-Value: 0.003989928963853392\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 481\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 482\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7868852459016393\n", + "Average of Other Ratios: 0.38333333333333336\n", + "T-Statistic: -10.324506821548738\n", + "P-Value: 0.0019381469069513462\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3432203389830508\n", + "T-Statistic: -4.253333333333336\n", + "P-Value: 0.023821477797343008\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 482\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 483\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.42288251366120216\n", + "T-Statistic: -3.622802747830676\n", + "P-Value: 0.036175989328214676\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3728813559322034\n", + "T-Statistic: -3.9191835884530852\n", + "P-Value: 0.029541363360540956\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 483\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 484\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.3319672131147541\n", + "T-Statistic: -17.615818436997262\n", + "P-Value: 0.0003987923532231844\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3247175141242938\n", + "T-Statistic: -3.2435043974379334\n", + "P-Value: 0.04772389993910521\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 484\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 485\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.4146174863387978\n", + "T-Statistic: -6.525168128577647\n", + "P-Value: 0.007313787597179912\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.385593220338983\n", + "T-Statistic: -3.2052128901777346\n", + "P-Value: 0.04913559685870657\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 485\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 486\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3689207650273224\n", + "T-Statistic: -4.354954591021095\n", + "P-Value: 0.022370006686486543\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3458333333333333\n", + "T-Statistic: -6.3744164409558755\n", + "P-Value: 0.007815387560659307\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 486\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 487\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.3625\n", + "T-Statistic: -4.5595686353563485\n", + "P-Value: 0.01977785439608666\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.3120056497175141\n", + "T-Statistic: -10.204926179291313\n", + "P-Value: 0.0020055260919182947\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 487\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 488\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.41475409836065574\n", + "T-Statistic: -8.217472162729653\n", + "P-Value: 0.00377202241458133\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.3757062146892655\n", + "T-Statistic: -2.741962532807021\n", + "P-Value: 0.07122087552292544\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 488\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 489\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.4316256830601093\n", + "T-Statistic: -6.118804569189945\n", + "P-Value: 0.008774285877481431\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.4221045197740113\n", + "T-Statistic: -5.546054501123849\n", + "P-Value: 0.011558137961560508\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 489\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 490\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.3958333333333333\n", + "T-Statistic: -5.135277089276766\n", + "P-Value: 0.014303932115525365\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.37951977401129944\n", + "T-Statistic: -4.250201094643769\n", + "P-Value: 0.02386812308620629\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 490\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 491\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4191256830601093\n", + "T-Statistic: -3.801261952080425\n", + "P-Value: 0.03197715417981016\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3728813559322034\n", + "T-Statistic: -4.984275273297009\n", + "P-Value: 0.015525883080703483\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 491\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 492\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -6.232543479694566\n", + "P-Value: 0.008329553737779013\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3332627118644068\n", + "T-Statistic: -7.640503978788062\n", + "P-Value: 0.004655333124225627\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 492\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 493\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -8.2470608782833\n", + "P-Value: 0.0037329364018606546\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.37570621468926557\n", + "T-Statistic: -6.286185570937117\n", + "P-Value: 0.008130066802486461\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 493\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 494\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.41892076502732245\n", + "T-Statistic: -5.9784922810858205\n", + "P-Value: 0.009366890251308508\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3500706214689266\n", + "T-Statistic: -2.435222247729202\n", + "P-Value: 0.09289556214456558\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 494\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 495\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4479508196721312\n", + "T-Statistic: -2.7046377161768436\n", + "P-Value: 0.0734969637824313\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.4049435028248588\n", + "T-Statistic: -5.1288742164266266\n", + "P-Value: 0.0143531474427786\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 495\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 496\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.38169398907103824\n", + "T-Statistic: -8.926429305995807\n", + "P-Value: 0.002965896425199081\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3389830508474576\n", + "T-Statistic: -18.452822728966602\n", + "P-Value: 0.00034730393994395734\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 496\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 497\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4208333333333333\n", + "T-Statistic: -6.853539958645125\n", + "P-Value: 0.00635920568434066\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.39505649717514124\n", + "T-Statistic: -3.8915662650602405\n", + "P-Value: 0.16012516703063404\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 497\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 498\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4351775956284153\n", + "T-Statistic: -3.62042968770528\n", + "P-Value: 0.03623643715847006\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.38799435028248586\n", + "T-Statistic: -4.361164169302447\n", + "P-Value: 0.022285069422793412\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 498\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 499\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -13.615808159190916\n", + "P-Value: 0.0008569802174432299\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.37973163841807916\n", + "T-Statistic: -2.7216754734382707\n", + "P-Value: 0.07244696363099003\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 499\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 500\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.3833333333333333\n", + "T-Statistic: -13.310561106566025\n", + "P-Value: 0.0009164879399678906\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3500706214689266\n", + "T-Statistic: -2.6495878836827105\n", + "P-Value: 0.07702097888753302\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 500\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 501\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.26666666666666666\n", + "T-Statistic: -13.732158696046199\n", + "P-Value: 0.0008356507564616465\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.2699152542372881\n", + "T-Statistic: -4.064774047804071\n", + "P-Value: 0.02685304092817379\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 501\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 502\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3980874316939891\n", + "T-Statistic: -8.506737943489776\n", + "P-Value: 0.003411830999011348\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.354590395480226\n", + "T-Statistic: -3.6417926698166023\n", + "P-Value: 0.035696879201596796\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 502\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 503\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.44166666666666665\n", + "T-Statistic: -5.438685446957326\n", + "P-Value: 0.01220408026603391\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3714689265536723\n", + "T-Statistic: -6.236810901332353\n", + "P-Value: 0.008313449992551876\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 503\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 504\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.4064890710382514\n", + "T-Statistic: -2.559704324213269\n", + "P-Value: 0.0832355641629743\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3586158192090395\n", + "T-Statistic: -4.619015880036581\n", + "P-Value: 0.019098463373393156\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 504\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 505\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.4148224043715847\n", + "T-Statistic: -5.693637680357018\n", + "P-Value: 0.010741337381016519\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.39661016949152544\n", + "T-Statistic: -2.9340578815309537\n", + "P-Value: 0.06080104286808068\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 505\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 506\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4025273224043716\n", + "T-Statistic: -8.890442589650126\n", + "P-Value: 0.0030009962178549105\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.31228813559322033\n", + "T-Statistic: -6.309916959963368\n", + "P-Value: 0.008043815258589415\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 506\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 507\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.42500000000000004\n", + "T-Statistic: -6.851133053826303\n", + "P-Value: 0.006365586871051235\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3627824858757063\n", + "T-Statistic: -2.7319309661026785\n", + "P-Value: 0.07182390621092707\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 507\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 508\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.4791666666666667\n", + "T-Statistic: -5.22212477804604\n", + "P-Value: 0.013657614104771973\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.3716101694915254\n", + "T-Statistic: -2.9643107979224634\n", + "P-Value: 0.059338263715170134\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 508\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 509\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3894808743169399\n", + "T-Statistic: -4.915294791461667\n", + "P-Value: 0.01612940842830176\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3601694915254237\n", + "T-Statistic: -3.409705862611856\n", + "P-Value: 0.04215978877786793\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 509\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 510\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8688524590163934\n", + "Average of Other Ratios: 0.35\n", + "T-Statistic: -25.41847552756969\n", + "P-Value: 0.0001335388647953177\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3630649717514124\n", + "T-Statistic: -5.7764663818628845\n", + "P-Value: 0.010315868020629015\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 510\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 511\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.37315573770491806\n", + "T-Statistic: -3.2713284175710178\n", + "P-Value: 0.04673018421334356\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.33742937853107347\n", + "T-Statistic: -6.023354344153489\n", + "P-Value: 0.009171855397206797\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 511\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 512\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.43606557377049177\n", + "T-Statistic: -5.107098757651382\n", + "P-Value: 0.014522185166915114\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3258003766478343\n", + "T-Statistic: -48.499999999999964\n", + "P-Value: 0.00042485397562817395\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 512\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 513\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.37315573770491806\n", + "T-Statistic: -8.428620891983224\n", + "P-Value: 0.003504451310362902\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.2867231638418079\n", + "T-Statistic: -4.840000000000001\n", + "P-Value: 0.016823522712679524\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 513\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 514\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.35273224043715845\n", + "T-Statistic: -29.617790180570413\n", + "P-Value: 8.45343765080844e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3209745762711864\n", + "T-Statistic: -3.5295270186824976\n", + "P-Value: 0.038652315352543654\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 514\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 515\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.44999999999999996\n", + "T-Statistic: -4.3195919820040585\n", + "P-Value: 0.02286178010818133\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3587570621468927\n", + "T-Statistic: -2.6549539521063013\n", + "P-Value: 0.07666840199371955\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 515\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 516\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.3291666666666667\n", + "T-Statistic: -15.836371260048626\n", + "P-Value: 0.000547399610373209\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.286864406779661\n", + "T-Statistic: -10.52321129742476\n", + "P-Value: 0.0018326710996097446\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 516\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 517\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.4333333333333333\n", + "T-Statistic: -2.604375511062208\n", + "P-Value: 0.08007263680217473\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.3799435028248587\n", + "T-Statistic: -6.86666666666667\n", + "P-Value: 0.006324551982225255\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 517\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 518\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.42500000000000004\n", + "T-Statistic: -4.639344262295081\n", + "P-Value: 0.01887310292166381\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3755649717514124\n", + "T-Statistic: -3.418861432750501\n", + "P-Value: 0.04187784075273277\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 518\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 519\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.3958333333333333\n", + "T-Statistic: -4.09708695591706\n", + "P-Value: 0.026299586172448756\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.29964689265536726\n", + "T-Statistic: -5.609714558498766\n", + "P-Value: 0.011196176065376182\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 519\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 520\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.3958333333333333\n", + "T-Statistic: -3.537066558867117\n", + "P-Value: 0.03844426851884796\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.32024482109227875\n", + "T-Statistic: -10.47368421052631\n", + "P-Value: 0.008993145222343412\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 520\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 521\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4230874316939891\n", + "T-Statistic: -6.814259023242736\n", + "P-Value: 0.006464409944966311\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3706214689265537\n", + "T-Statistic: -6.849973076409931\n", + "P-Value: 0.02065393933365802\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 521\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 522\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.3416666666666666\n", + "T-Statistic: -30.71720234254342\n", + "P-Value: 7.580035597644182e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3601694915254237\n", + "T-Statistic: -3.1333333333333346\n", + "P-Value: 0.05193110106234139\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 522\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 523\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.42916666666666664\n", + "T-Statistic: -9.51209634369107\n", + "P-Value: 0.002463921302178941\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3940677966101695\n", + "T-Statistic: -5.245148133977578\n", + "P-Value: 0.013492692472278441\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 523\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 524\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.39772313296903467\n", + "T-Statistic: -11.857859091548148\n", + "P-Value: 0.00703694821787774\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.37175141242937854\n", + "T-Statistic: -2.224480146264155\n", + "P-Value: 0.11257283375371135\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 524\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 525\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.36461748633879776\n", + "T-Statistic: -8.164949484411201\n", + "P-Value: 0.003842746397948636\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.33340395480225987\n", + "T-Statistic: -4.859171643105984\n", + "P-Value: 0.016643155194105735\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 525\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 526\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.5\n", + "T-Statistic: -2.0077784776911325\n", + "P-Value: 0.1382803772888902\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.4194915254237288\n", + "T-Statistic: -2.8609898912604983\n", + "P-Value: 0.06452459428804105\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 526\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 527\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.4025273224043716\n", + "T-Statistic: -24.399378398771418\n", + "P-Value: 0.0001509087662519508\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.33333333333333337\n", + "T-Statistic: -6.275716324421885\n", + "P-Value: 0.008168503479790335\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 527\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 528\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.40211748633879785\n", + "T-Statistic: -3.4850713293771878\n", + "P-Value: 0.03990865507980857\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.29138418079096046\n", + "T-Statistic: -5.949008298622876\n", + "P-Value: 0.009498040523452093\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 528\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 529\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.4541666666666667\n", + "T-Statistic: -2.176256072652707\n", + "P-Value: 0.11776205733944392\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3882768361581921\n", + "T-Statistic: -4.096759822564065\n", + "P-Value: 0.026305115270877533\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 529\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 530\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.39166666666666666\n", + "T-Statistic: -4.846843823072969\n", + "P-Value: 0.01675884522410491\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3389830508474576\n", + "T-Statistic: -2.4666312377251187\n", + "P-Value: 0.09033399701898738\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 530\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 531\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.4689890710382514\n", + "T-Statistic: -2.3029347384916004\n", + "P-Value: 0.10470629811332137\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.39216101694915256\n", + "T-Statistic: -2.6138797970211702\n", + "P-Value: 0.07941891525657455\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 531\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 532\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.3125\n", + "T-Statistic: -40.55023240686837\n", + "P-Value: 3.3002033131333967e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.2865348399246704\n", + "T-Statistic: -10.417426280656978\n", + "P-Value: 0.009089216213235031\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 532\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 533\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.44330601092896177\n", + "T-Statistic: -3.289205567147825\n", + "P-Value: 0.04610551507892302\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.40494350282485875\n", + "T-Statistic: -3.5632864433698304\n", + "P-Value: 0.03773178159693135\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 533\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 534\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3819672131147541\n", + "T-Statistic: -2.224352555510678\n", + "P-Value: 0.11258619639823252\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3081214689265537\n", + "T-Statistic: -2.706180513915112\n", + "P-Value: 0.07340111077355108\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 534\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 535\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.38333333333333336\n", + "T-Statistic: -2.615568260013164\n", + "P-Value: 0.07930346961482511\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3375706214689265\n", + "T-Statistic: -3.298098944452524\n", + "P-Value: 0.04579869976521974\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 535\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 536\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7666666666666667\n", + "Average of Other Ratios: 0.3896857923497268\n", + "T-Statistic: -13.06989618982596\n", + "P-Value: 0.0009673345468333082\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.30374293785310735\n", + "T-Statistic: -3.6378487523667777\n", + "P-Value: 0.0357957127768318\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 536\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 537\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.44999999999999996\n", + "T-Statistic: -2.9350727376464576\n", + "P-Value: 0.06075124544604779\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.37980225988700567\n", + "T-Statistic: -5.144656775656592\n", + "P-Value: 0.014232232343107012\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 537\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 538\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.38333333333333336\n", + "T-Statistic: -4.592341249903026\n", + "P-Value: 0.0193995061866707\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3334039548022599\n", + "T-Statistic: -5.550438587280521\n", + "P-Value: 0.011532723915216122\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 538\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 539\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3729508196721311\n", + "T-Statistic: -5.094565492831203\n", + "P-Value: 0.014620656170024945\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3456214689265537\n", + "T-Statistic: -2.619445228666292\n", + "P-Value: 0.07903917160851617\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 539\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 540\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7666666666666667\n", + "Average of Other Ratios: 0.34446721311475414\n", + "T-Statistic: -16.788036508322786\n", + "P-Value: 0.00046020497513463827\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.34597457627118644\n", + "T-Statistic: -22.472055097835053\n", + "P-Value: 0.0001929545430821344\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 540\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 541\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.33333333333333337\n", + "T-Statistic: -5.392319340084748\n", + "P-Value: 0.012497618930811919\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.28707627118644075\n", + "T-Statistic: -3.9963952715287125\n", + "P-Value: 0.02807461925885932\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 541\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 542\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.41666666666666663\n", + "T-Statistic: -12.480891064461987\n", + "P-Value: 0.0011086313738353398\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3855932203389831\n", + "T-Statistic: -3.388738579570791\n", + "P-Value: 0.042814628734577545\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 542\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 543\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.45\n", + "T-Statistic: -15.07113703400236\n", + "P-Value: 0.0006341498821599429\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.4495291902071563\n", + "T-Statistic: -3.306335305824179\n", + "P-Value: 0.08057456813744839\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 543\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 544\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.38592896174863384\n", + "T-Statistic: -7.972344974800964\n", + "P-Value: 0.004117606574879096\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.33326271186440676\n", + "T-Statistic: -4.308228721563899\n", + "P-Value: 0.023022768426326834\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 544\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 545\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.3791666666666667\n", + "T-Statistic: -7.383406964717152\n", + "P-Value: 0.00513735130962263\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.30409604519774014\n", + "T-Statistic: -3.009193969690081\n", + "P-Value: 0.057248412075778514\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 545\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 546\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -3.4195951141068646\n", + "P-Value: 0.041855350747093845\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.29526836158192094\n", + "T-Statistic: -7.3968729657562395\n", + "P-Value: 0.005110511890500865\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 546\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 547\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.48750000000000004\n", + "T-Statistic: -2.6937228618928075\n", + "P-Value: 0.07417954696153008\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3627824858757062\n", + "T-Statistic: -4.484419263456091\n", + "P-Value: 0.020682260794596535\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 547\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 548\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.3375\n", + "T-Statistic: -8.980327868852457\n", + "P-Value: 0.0029143354515137163\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3432203389830508\n", + "T-Statistic: -8.529366637570185\n", + "P-Value: 0.0033856068172450396\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 548\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 549\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.3610655737704918\n", + "T-Statistic: -6.352580462402448\n", + "P-Value: 0.007891754845695532\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.3163135593220339\n", + "T-Statistic: -7.643205738383732\n", + "P-Value: 0.004650591457090468\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 549\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 550\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3734972677595629\n", + "T-Statistic: -4.334361862984616\n", + "P-Value: 0.022654697680077447\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.2828389830508474\n", + "T-Statistic: -3.1258403580998007\n", + "P-Value: 0.052233941204207116\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 550\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 551\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4599726775956284\n", + "T-Statistic: -2.514156166273152\n", + "P-Value: 0.08662037793756441\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.39406779661016955\n", + "T-Statistic: -2.982530505374287\n", + "P-Value: 0.058478563965002554\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 551\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 552\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.360724043715847\n", + "T-Statistic: -3.4306011747993215\n", + "P-Value: 0.041519810852172816\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.30903954802259886\n", + "T-Statistic: -7.0399681163437675\n", + "P-Value: 0.019586257511572443\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 552\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 553\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.44405737704918036\n", + "T-Statistic: -3.2820666584674085\n", + "P-Value: 0.04635368887671906\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.38813559322033897\n", + "T-Statistic: -1.8603721116372136\n", + "P-Value: 0.15977096644438923\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 553\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 554\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.4273224043715847\n", + "T-Statistic: -6.912582855578067\n", + "P-Value: 0.006205274596049662\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.358545197740113\n", + "T-Statistic: -4.751834676331621\n", + "P-Value: 0.017686518806967016\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 554\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 555\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.39849726775956285\n", + "T-Statistic: -7.109187347867667\n", + "P-Value: 0.005726771511083016\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3206920903954802\n", + "T-Statistic: -3.269035331124903\n", + "P-Value: 0.046811083782206005\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 555\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 556\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.3875\n", + "T-Statistic: -7.23666133386256\n", + "P-Value: 0.005442256485075891\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3500706214689266\n", + "T-Statistic: -3.7595621188342796\n", + "P-Value: 0.032899914767886236\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 556\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 557\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.41523224043715845\n", + "T-Statistic: -2.490304414512709\n", + "P-Value: 0.08846028307415578\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3628531073446328\n", + "T-Statistic: -5.912181117649445\n", + "P-Value: 0.009665258648047835\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 557\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 558\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.4265710382513661\n", + "T-Statistic: -3.6163076187603744\n", + "P-Value: 0.03634174375169587\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.38834745762711864\n", + "T-Statistic: -3.4214826476365072\n", + "P-Value: 0.04179756150788246\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 558\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 559\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.35416666666666663\n", + "T-Statistic: -13.436668405820718\n", + "P-Value: 0.000891257804744702\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.35169491525423724\n", + "T-Statistic: -4.748264132284415\n", + "P-Value: 0.017722668444947104\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 559\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 560\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3236338797814208\n", + "T-Statistic: -3.600554934819319\n", + "P-Value: 0.03674779775320059\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.274364406779661\n", + "T-Statistic: -4.35035650844239\n", + "P-Value: 0.022433171283776563\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 560\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 561\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.2903688524590164\n", + "T-Statistic: -7.987447429719284\n", + "P-Value: 0.004095133323667272\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.19851694915254237\n", + "T-Statistic: -3.3448722836047113\n", + "P-Value: 0.04422697438835633\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 561\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 562\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.45833333333333337\n", + "T-Statistic: -5.720862788355775\n", + "P-Value: 0.010598994876723512\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.41370056497175145\n", + "T-Statistic: -3.4764271828366216\n", + "P-Value: 0.04015895679106413\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 562\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 563\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -5.605717509713631\n", + "P-Value: 0.01121846256816809\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.341454802259887\n", + "T-Statistic: -2.2689902462968896\n", + "P-Value: 0.10802564058574593\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 563\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 564\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3857923497267759\n", + "T-Statistic: -2.773401460624423\n", + "P-Value: 0.06937128230425361\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.37153954802259886\n", + "T-Statistic: -3.9815118119797472\n", + "P-Value: 0.028349938061132722\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 564\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 565\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.44583333333333336\n", + "T-Statistic: -8.401275011504453\n", + "P-Value: 0.0035376591875335185\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3755649717514124\n", + "T-Statistic: -6.27344916163044\n", + "P-Value: 0.008176858545156238\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 565\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 566\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.39583333333333337\n", + "T-Statistic: -5.442971730211302\n", + "P-Value: 0.012177401351288936\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3840395480225989\n", + "T-Statistic: -9.780487804878028\n", + "P-Value: 0.002271338003074173\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 566\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 567\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.41250000000000003\n", + "T-Statistic: -4.643102959582609\n", + "P-Value: 0.018831812762783767\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.35451977401129947\n", + "T-Statistic: -4.066666666666665\n", + "P-Value: 0.02682021272712732\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 567\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 568\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.45614754098360655\n", + "T-Statistic: -3.6102043743828824\n", + "P-Value: 0.03649838253815035\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.4050141242937853\n", + "T-Statistic: -3.35873852777973\n", + "P-Value: 0.0437742186193112\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 568\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 569\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.4227459016393443\n", + "T-Statistic: -5.2448185798443125\n", + "P-Value: 0.013495034751188689\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3879943502824859\n", + "T-Statistic: -4.361164169302444\n", + "P-Value: 0.02228506942279346\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 569\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 570\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.425\n", + "T-Statistic: -4.312990302477515\n", + "P-Value: 0.022955131152546198\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3421610169491526\n", + "T-Statistic: -3.2068965517241366\n", + "P-Value: 0.049072424069962796\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 570\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 571\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.4541666666666667\n", + "T-Statistic: -6.130104668195612\n", + "P-Value: 0.008728726955771456\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.405225988700565\n", + "T-Statistic: -3.4882407793268584\n", + "P-Value: 0.03981737779020827\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 571\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 572\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.4685109289617486\n", + "T-Statistic: -6.044103218158989\n", + "P-Value: 0.009083452194363606\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.576271186440678\n", + "Average of Other Ratios: 0.4471751412429379\n", + "T-Statistic: -6.645559382413252\n", + "P-Value: 0.006943215258999799\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 572\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 573\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.551844262295082\n", + "T-Statistic: -16.28766880858878\n", + "P-Value: 0.0005035373682223364\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.576271186440678\n", + "Average of Other Ratios: 0.4717514124293785\n", + "T-Statistic: -6.08276253029822\n", + "P-Value: 0.02597846598858569\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 573\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 574\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -4.923541270396146\n", + "P-Value: 0.016055668617949045\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.2995762711864407\n", + "T-Statistic: -10.972170265475434\n", + "P-Value: 0.0016209009976600197\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 574\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 575\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.4148224043715847\n", + "T-Statistic: -2.935331303657975\n", + "P-Value: 0.06073856617509204\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.33145009416195853\n", + "T-Statistic: -2.8168113585237156\n", + "P-Value: 0.10631108992737348\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 575\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 576\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.39788251366120214\n", + "T-Statistic: -3.995817683425296\n", + "P-Value: 0.028085239330913513\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3713983050847458\n", + "T-Statistic: -3.303424358920434\n", + "P-Value: 0.045616214777893696\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 576\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 577\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.42083333333333334\n", + "T-Statistic: -5.615165491162119\n", + "P-Value: 0.011165876393712151\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3542372881355932\n", + "T-Statistic: -9.208061462894172\n", + "P-Value: 0.0027091139233435414\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 577\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 578\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.44760928961748636\n", + "T-Statistic: -2.8779672917258896\n", + "P-Value: 0.06363455777153253\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.35007062146892653\n", + "T-Statistic: -8.932852690514894\n", + "P-Value: 0.0029596885792663864\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 578\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 579\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.4125\n", + "T-Statistic: -4.800754235493788\n", + "P-Value: 0.017200753129888367\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3626412429378531\n", + "T-Statistic: -3.365301828336874\n", + "P-Value: 0.04356196753364835\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 579\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 580\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.33749999999999997\n", + "T-Statistic: -6.629461654119429\n", + "P-Value: 0.006991312298326812\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.2785310734463277\n", + "T-Statistic: -5.283646176355578\n", + "P-Value: 0.013222684385745894\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 580\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 581\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -10.98401471759805\n", + "P-Value: 0.0016157647819501492\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3516949152542373\n", + "T-Statistic: -4.489140335563251\n", + "P-Value: 0.02062388445422241\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 581\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 582\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4559426229508197\n", + "T-Statistic: -2.024940270686622\n", + "P-Value: 0.13600606005905447\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.40098870056497177\n", + "T-Statistic: -7.988978716899146\n", + "P-Value: 0.004092863743178219\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 582\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 583\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.36878415300546447\n", + "T-Statistic: -2.8875369089248184\n", + "P-Value: 0.06313962819005703\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.28255649717514125\n", + "T-Statistic: -6.786220279071329\n", + "P-Value: 0.0065409101032541635\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 583\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 584\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3898224043715847\n", + "T-Statistic: -6.736784395106943\n", + "P-Value: 0.006678717888232159\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.33771186440677964\n", + "T-Statistic: -5.3858337188310035\n", + "P-Value: 0.012539411668401262\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 584\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 585\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -15.59782388553334\n", + "P-Value: 0.0005726503128458851\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.38757062146892657\n", + "T-Statistic: -2.4285714285714284\n", + "P-Value: 0.13584143478190688\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 585\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 586\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.35416666666666663\n", + "T-Statistic: -10.11025517993969\n", + "P-Value: 0.002061088422236367\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3417372881355932\n", + "T-Statistic: -6.45766024002191\n", + "P-Value: 0.007533010897429456\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 586\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 587\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.43135245901639346\n", + "T-Statistic: -8.36023757475581\n", + "P-Value: 0.0035882787219289397\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3964689265536723\n", + "T-Statistic: -4.172537109479583\n", + "P-Value: 0.02506338811524026\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 587\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 588\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.40683060109289615\n", + "T-Statistic: -3.685355043522012\n", + "P-Value: 0.034628080098915076\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3169491525423729\n", + "T-Statistic: -1.9629629629629635\n", + "P-Value: 0.14443412236891323\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 588\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 589\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -8.342713328952133\n", + "P-Value: 0.0036101866302316646\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.364406779661017\n", + "T-Statistic: -6.301611145596256\n", + "P-Value: 0.008073865317362929\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 589\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 590\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -22.125683060109264\n", + "P-Value: 0.00020211389888430634\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3463983050847458\n", + "T-Statistic: -2.043874646199097\n", + "P-Value: 0.13354795806792266\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 590\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 591\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.40204918032786885\n", + "T-Statistic: -3.8720368025363827\n", + "P-Value: 0.030485627570044506\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3966101694915254\n", + "T-Statistic: -3.050041359322525\n", + "P-Value: 0.05542620511793901\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 591\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 592\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.41509562841530057\n", + "T-Statistic: -3.918936678185741\n", + "P-Value: 0.029546209898049555\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3632062146892655\n", + "T-Statistic: -2.9222455629486523\n", + "P-Value: 0.06138443122939498\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 592\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 593\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -2.2816691505216085\n", + "P-Value: 0.10677112400407492\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3459745762711864\n", + "T-Statistic: -3.610360849034176\n", + "P-Value: 0.03649435588128962\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 593\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 594\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4145491803278688\n", + "T-Statistic: -3.225298064940852\n", + "P-Value: 0.04838862749274675\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.33785310734463275\n", + "T-Statistic: -2.653428192571171\n", + "P-Value: 0.07676844842248204\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 594\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 595\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.3833333333333333\n", + "T-Statistic: -5.3934540699330284\n", + "P-Value: 0.012490325558165266\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.346045197740113\n", + "T-Statistic: -3.3888747468281344\n", + "P-Value: 0.04281033448755565\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 595\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 596\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.3523224043715847\n", + "T-Statistic: -6.072607039010736\n", + "P-Value: 0.008963822063323797\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.3305084745762712\n", + "T-Statistic: -2.6185237571323796\n", + "P-Value: 0.07910189090563483\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 596\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 597\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4125\n", + "T-Statistic: -6.486338797814206\n", + "P-Value: 0.007438839601531942\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3377118644067797\n", + "T-Statistic: -6.269358747755155\n", + "P-Value: 0.00819196116993917\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 597\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 598\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.4363387978142077\n", + "T-Statistic: -3.468352695270501\n", + "P-Value: 0.07401784897865982\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.38389830508474576\n", + "T-Statistic: -8.48622665150705\n", + "P-Value: 0.0034358339331581286\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 598\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 599\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8333333333333334\n", + "Average of Other Ratios: 0.4443306010928962\n", + "T-Statistic: -13.069044138229902\n", + "P-Value: 0.0009675211431764483\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.4096045197740113\n", + "T-Statistic: -4.528095585898831\n", + "P-Value: 0.020150257453059105\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 599\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 600\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.41250000000000003\n", + "T-Statistic: -3.6410524102519206\n", + "P-Value: 0.03571540330483068\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3882062146892655\n", + "T-Statistic: -4.997867054186407\n", + "P-Value: 0.015410451099079129\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 600\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 601\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.194603825136612\n", + "T-Statistic: -7.079275684498563\n", + "P-Value: 0.005796358119040252\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.12245762711864408\n", + "T-Statistic: -3.608916325424141\n", + "P-Value: 0.03653155025346661\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 601\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 602\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.2666666666666667\n", + "T-Statistic: -13.671082075940005\n", + "P-Value: 0.0008467582451760971\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.24032485875706217\n", + "T-Statistic: -3.0165616144994942\n", + "P-Value: 0.05691423813610457\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 602\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 603\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.3041666666666667\n", + "T-Statistic: -3.470708937267535\n", + "P-Value: 0.04032563476307778\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.20247175141242937\n", + "T-Statistic: -11.096292845694657\n", + "P-Value: 0.00156813241982992\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 603\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 604\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.2525273224043716\n", + "T-Statistic: -2.709448750044224\n", + "P-Value: 0.07319856881186551\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.14759887005649716\n", + "T-Statistic: -2.8959560191449962\n", + "P-Value: 0.06270817363169148\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 604\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 605\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.21666666666666665\n", + "T-Statistic: -5.554402221351035\n", + "P-Value: 0.011509810078003106\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1694915254237288\n", + "Average of Other Ratios: 0.1265536723163842\n", + "T-Statistic: -3.972583638518804\n", + "P-Value: 0.02851676236015958\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 605\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 606\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.26939890710382514\n", + "T-Statistic: -3.323461871958838\n", + "P-Value: 0.04493779701217889\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.13933615819209041\n", + "T-Statistic: -4.2672442091017775\n", + "P-Value: 0.02361573435774843\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 606\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 607\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.24583333333333332\n", + "T-Statistic: -7.477098647437464\n", + "P-Value: 0.004954385859733728\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.15188323917137478\n", + "T-Statistic: -3.3981981491560638\n", + "P-Value: 0.0767589601591142\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 607\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 608\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.35416666666666663\n", + "T-Statistic: -3.5645342704092955\n", + "P-Value: 0.03769829534729827\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3333333333333333\n", + "Average of Other Ratios: 0.2584745762711864\n", + "T-Statistic: -5.172270386627225\n", + "P-Value: 0.014023853313803805\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 608\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 609\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.2655737704918033\n", + "T-Statistic: -4.795356126981156\n", + "P-Value: 0.017253497348406005\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.17711864406779662\n", + "T-Statistic: -5.84274742881549\n", + "P-Value: 0.00999117759634526\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 609\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 610\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.2583333333333333\n", + "T-Statistic: -8.132012676182216\n", + "P-Value: 0.0038879932294071462\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.25\n", + "Average of Other Ratios: 0.1694915254237288\n", + "T-Statistic: -3.878358759406699\n", + "P-Value: 0.03035678137281933\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 610\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 611\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.39344262295081966\n", + "Average of Other Ratios: 0.21666666666666667\n", + "T-Statistic: -9.819764009062963\n", + "P-Value: 0.0022448478763413245\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.13057909604519774\n", + "T-Statistic: -4.374936505877199\n", + "P-Value: 0.02209816935791197\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 611\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 612\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.26666666666666666\n", + "T-Statistic: -2.7725030413659453\n", + "P-Value: 0.06942330291539806\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.18128531073446327\n", + "T-Statistic: -4.128224087608492\n", + "P-Value: 0.025780103759508587\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 612\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 613\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.21147540983606558\n", + "T-Statistic: -10.383401644446371\n", + "P-Value: 0.001906063349217342\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1694915254237288\n", + "Average of Other Ratios: 0.10677966101694915\n", + "T-Statistic: -2.53917501033781\n", + "P-Value: 0.12636332880012854\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 613\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 614\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.19166666666666668\n", + "T-Statistic: -22.535357185812448\n", + "P-Value: 0.00019134068124253585\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.1518361581920904\n", + "T-Statistic: -3.0820624157214693\n", + "P-Value: 0.054048493565592826\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 614\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 615\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.20416666666666666\n", + "T-Statistic: -12.92280294143108\n", + "P-Value: 0.0010002691021999664\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.15254237288135594\n", + "Average of Other Ratios: 0.12648305084745762\n", + "T-Statistic: -2.492276285625687\n", + "P-Value: 0.0883063687998794\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 615\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 616\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.25416666666666665\n", + "T-Statistic: -11.210456983480942\n", + "P-Value: 0.001521591601458421\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.21101694915254238\n", + "T-Statistic: -4.841386618546792\n", + "P-Value: 0.016810392289180406\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 616\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 617\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.21666666666666667\n", + "T-Statistic: -15.150632180505205\n", + "P-Value: 0.0006243223321137057\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.13465160075329566\n", + "T-Statistic: -2.718662067942085\n", + "P-Value: 0.11285123312157036\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 617\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 618\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.22499999999999998\n", + "T-Statistic: -6.918738471764206\n", + "P-Value: 0.006189509895466707\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.1602401129943503\n", + "T-Statistic: -2.531317735884927\n", + "P-Value: 0.08532554275225189\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 618\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 619\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.2528688524590164\n", + "T-Statistic: -8.617015849137902\n", + "P-Value: 0.0032865107935262\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.19406779661016949\n", + "T-Statistic: -8.748025509254019\n", + "P-Value: 0.0031454221399862072\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 619\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 620\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.23620218579234975\n", + "T-Statistic: -5.140402367328491\n", + "P-Value: 0.014264695462063416\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.15176553672316384\n", + "T-Statistic: -5.734186911097393\n", + "P-Value: 0.01053023168597254\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 620\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 621\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.2791666666666667\n", + "T-Statistic: -9.365095173805795\n", + "P-Value: 0.0025786301461165144\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.25296610169491524\n", + "T-Statistic: -2.7041039897995947\n", + "P-Value: 0.07353015997518966\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 621\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 622\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.22916666666666663\n", + "T-Statistic: -8.306260504867486\n", + "P-Value: 0.003656327731735171\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.15190677966101696\n", + "T-Statistic: -7.409133468514449\n", + "P-Value: 0.005086235812941152\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 622\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 623\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.2375\n", + "T-Statistic: -7.720607509620097\n", + "P-Value: 0.004517414616084237\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.15254237288135594\n", + "T-Statistic: -3.5204166420089162\n", + "P-Value: 0.038905627181690466\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 623\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 624\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7540983606557377\n", + "Average of Other Ratios: 0.15833333333333333\n", + "T-Statistic: -14.492905117377825\n", + "P-Value: 0.0007122141225715405\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1899717514124294\n", + "T-Statistic: -5.4166289591447665\n", + "P-Value: 0.012342580506632655\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 624\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 625\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4098360655737705\n", + "Average of Other Ratios: 0.23333333333333334\n", + "T-Statistic: -8.20310571032784\n", + "P-Value: 0.0037911953280657033\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.1435734463276836\n", + "T-Statistic: -5.128214329323895\n", + "P-Value: 0.01435823217533278\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 625\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 626\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.2568989071038251\n", + "T-Statistic: -6.954617925271148\n", + "P-Value: 0.006098663645228381\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.16864406779661018\n", + "T-Statistic: -6.5059819266483885\n", + "P-Value: 0.00737522940630693\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 626\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 627\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.22916666666666669\n", + "T-Statistic: -17.17085230252723\n", + "P-Value: 0.00043034591455487535\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.20338983050847456\n", + "T-Statistic: -2.4984439601924695\n", + "P-Value: 0.08782706589330419\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 627\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 628\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.2658469945355191\n", + "T-Statistic: -2.8102104403444548\n", + "P-Value: 0.06728103964001445\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.18559322033898307\n", + "T-Statistic: -8.275546059124256\n", + "P-Value: 0.0036958123216244147\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 628\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 629\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.5185109289617487\n", + "T-Statistic: -7.578250897242925\n", + "P-Value: 0.004766379016909529\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.48107344632768356\n", + "T-Statistic: -2.69383410385129\n", + "P-Value: 0.07417255068242208\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 629\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 630\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.75\n", + "Average of Other Ratios: 0.19460382513661204\n", + "T-Statistic: -10.85330590771855\n", + "P-Value: 0.0016736652012390987\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.1643361581920904\n", + "T-Statistic: -5.023778124999512\n", + "P-Value: 0.015193473744699431\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 630\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 631\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.23203551912568304\n", + "T-Statistic: -2.7592753489500113\n", + "P-Value: 0.07019486191792593\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.15254237288135594\n", + "Average of Other Ratios: 0.11233521657250471\n", + "T-Statistic: -2.7098687752823145\n", + "P-Value: 0.11346466393886437\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 631\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 632\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.27404371584699455\n", + "T-Statistic: -3.6739700755121674\n", + "P-Value: 0.03490341382115382\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.1689265536723164\n", + "T-Statistic: -5.86064261618386\n", + "P-Value: 0.00990581703436056\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 632\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 633\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.2650273224043716\n", + "T-Statistic: -2.3392495841230168\n", + "P-Value: 0.10129085666135956\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.15600282485875705\n", + "T-Statistic: -3.1321496288779005\n", + "P-Value: 0.051978794780651254\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 633\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 634\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.2791666666666667\n", + "T-Statistic: -4.2816691505216085\n", + "P-Value: 0.023404806469187092\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.17281073446327683\n", + "T-Statistic: -2.714263880607111\n", + "P-Value: 0.07290142241990857\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 634\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 635\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.24487704918032788\n", + "T-Statistic: -16.399313267142833\n", + "P-Value: 0.0004934130615457107\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.16878531073446326\n", + "T-Statistic: -4.294117647058823\n", + "P-Value: 0.023224732577996746\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 635\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 636\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.25314207650273224\n", + "T-Statistic: -4.511368522252938\n", + "P-Value: 0.02035188309309117\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2\n", + "Average of Other Ratios: 0.17372881355932202\n", + "T-Statistic: -6.200000000000003\n", + "P-Value: 0.008453719117567243\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 636\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 637\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.42500000000000004\n", + "T-Statistic: -7.254802469447766\n", + "P-Value: 0.005403292565475987\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.3711158192090396\n", + "T-Statistic: -5.424838731783427\n", + "P-Value: 0.01229078962720508\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 637\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 638\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.2614071038251366\n", + "T-Statistic: -6.303666365112512\n", + "P-Value: 0.008066415912528251\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.18566384180790962\n", + "T-Statistic: -4.932949247691678\n", + "P-Value: 0.01597207780336412\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 638\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 639\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.26249999999999996\n", + "T-Statistic: -4.999367050456228\n", + "P-Value: 0.015397780557052671\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.26666666666666666\n", + "Average of Other Ratios: 0.19491525423728812\n", + "T-Statistic: -6.558251799577894\n", + "P-Value: 0.0072094124531609394\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 639\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 640\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.2284153005464481\n", + "T-Statistic: -14.21620797979023\n", + "P-Value: 0.0007541140145429063\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1694915254237288\n", + "Average of Other Ratios: 0.12662429378531073\n", + "T-Statistic: -4.954648074701186\n", + "P-Value: 0.01578143114842512\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 640\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 641\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.24166666666666667\n", + "T-Statistic: -17.24695131862833\n", + "P-Value: 0.0004247197725637856\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.18135593220338986\n", + "T-Statistic: -7.923076923076922\n", + "P-Value: 0.004192062411449591\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 641\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 642\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.23210382513661199\n", + "T-Statistic: -5.707259199146484\n", + "P-Value: 0.010669808386105378\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.1398305084745763\n", + "T-Statistic: -4.195254538472044\n", + "P-Value: 0.02470591794360766\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 642\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 643\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.24446721311475408\n", + "T-Statistic: -3.2958071028221907\n", + "P-Value: 0.045877518615205004\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.1730225988700565\n", + "T-Statistic: -4.389315939938801\n", + "P-Value: 0.021905189439865372\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 643\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 644\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.2666666666666667\n", + "T-Statistic: -11.653770786981173\n", + "P-Value: 0.0013573049832111713\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.17718926553672315\n", + "T-Statistic: -9.34117796358571\n", + "P-Value: 0.0025979582105049733\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 644\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 645\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3770491803278688\n", + "Average of Other Ratios: 0.24583333333333335\n", + "T-Statistic: -4.076914857521508\n", + "P-Value: 0.02664334911823053\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.13483992467043315\n", + "T-Statistic: -8.711460465745363\n", + "P-Value: 0.012922185254776281\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 645\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 646\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.2333333333333333\n", + "T-Statistic: -4.863758570369447\n", + "P-Value: 0.016600374637456124\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.1812853107344633\n", + "T-Statistic: -2.856098645795084\n", + "P-Value: 0.06478389664989048\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 646\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 647\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.2\n", + "T-Statistic: -6.147724135202747\n", + "P-Value: 0.008658307683879244\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1694915254237288\n", + "Average of Other Ratios: 0.1266949152542373\n", + "T-Statistic: -3.796706309892804\n", + "P-Value: 0.032076336335065445\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 647\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 648\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.23333333333333334\n", + "T-Statistic: -17.08972839908021\n", + "P-Value: 0.00043645329683166667\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.18333333333333332\n", + "Average of Other Ratios: 0.13559322033898308\n", + "T-Statistic: -3.983368200684214\n", + "P-Value: 0.02831540894214516\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 648\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 649\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.2583333333333333\n", + "T-Statistic: -6.667849243203331\n", + "P-Value: 0.006877333446979579\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.16913841807909605\n", + "T-Statistic: -2.2349882150268345\n", + "P-Value: 0.11147884166525937\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 649\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 650\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.23333333333333334\n", + "T-Statistic: -9.186157399702243\n", + "P-Value: 0.0027280043187776987\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.19858757062146892\n", + "T-Statistic: -2.5535821728149255\n", + "P-Value: 0.08368093910992941\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 650\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 651\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.4375\n", + "T-Statistic: -1.9737703484788938\n", + "P-Value: 0.14292112743640065\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.39653954802259883\n", + "T-Statistic: -2.821061725972151\n", + "P-Value: 0.06667985231722355\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 651\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 652\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.2614071038251366\n", + "T-Statistic: -5.835105865583961\n", + "P-Value: 0.010027921838910365\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.22365819209039547\n", + "T-Statistic: -6.611650272701572\n", + "P-Value: 0.007045041561568846\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 652\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 653\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.18640710382513662\n", + "T-Statistic: -8.230490181795378\n", + "P-Value: 0.0037547595934809518\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.15254237288135594\n", + "Average of Other Ratios: 0.1290960451977401\n", + "T-Statistic: -1.6375555461118865\n", + "P-Value: 0.24316808629278763\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 653\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 654\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.2583333333333333\n", + "T-Statistic: -15.901213347642667\n", + "P-Value: 0.0005407927864783102\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.19830508474576272\n", + "T-Statistic: -5.461092327709232\n", + "P-Value: 0.01206545301607257\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 654\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 655\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.2823087431693989\n", + "T-Statistic: -10.150428532833908\n", + "P-Value: 0.0020372645895073123\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.16927966101694913\n", + "T-Statistic: -3.519195236968839\n", + "P-Value: 0.038939748753238246\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 655\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 656\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.23750000000000002\n", + "T-Statistic: -6.178438384650274\n", + "P-Value: 0.008537325917522763\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1772598870056497\n", + "T-Statistic: -6.871842709362766\n", + "P-Value: 0.00631095591061526\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 656\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 657\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.27083333333333337\n", + "T-Statistic: -3.3626410531598294\n", + "P-Value: 0.04364785672227368\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.21094632768361582\n", + "T-Statistic: -3.6366923315065693\n", + "P-Value: 0.035824758686131074\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 657\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 658\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3445355191256831\n", + "T-Statistic: -1.8802591489231106\n", + "P-Value: 0.15665493572520436\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.2918079096045198\n", + "T-Statistic: -2.173368590328213\n", + "P-Value: 0.16182504545842755\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 658\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 659\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.25416666666666665\n", + "T-Statistic: -4.330653825520752\n", + "P-Value: 0.022706457712737285\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2833333333333333\n", + "Average of Other Ratios: 0.1610169491525424\n", + "T-Statistic: -3.7266706420173588\n", + "P-Value: 0.033651949508593376\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 659\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 660\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36065573770491804\n", + "Average of Other Ratios: 0.30416666666666664\n", + "T-Statistic: -3.0571045827459344\n", + "P-Value: 0.05511854521199261\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.18524011299435028\n", + "T-Statistic: -3.1980807339949875\n", + "P-Value: 0.04940434566953938\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 660\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 661\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.24023224043715846\n", + "T-Statistic: -3.433838233132664\n", + "P-Value: 0.0414217747379\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.1602401129943503\n", + "T-Statistic: -4.406927169154064\n", + "P-Value: 0.02167179821796596\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 661\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 662\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3770491803278688\n", + "Average of Other Ratios: 0.24166666666666664\n", + "T-Statistic: -2.7999079904008193\n", + "P-Value: 0.06785808116951951\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.16468926553672317\n", + "T-Statistic: -8.002907374821234\n", + "P-Value: 0.004072295634170163\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 662\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 663\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.2189207650273224\n", + "T-Statistic: -2.8547072311071635\n", + "P-Value: 0.06485789803329635\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.13919491525423727\n", + "T-Statistic: -8.229362053239296\n", + "P-Value: 0.003756251434707851\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 663\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 664\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.4708333333333333\n", + "T-Statistic: -2.930782830598391\n", + "P-Value: 0.06096209353124864\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.44067796610169496\n", + "T-Statistic: -2.3309944988213394\n", + "P-Value: 0.10205524591620584\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 664\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 665\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.24166666666666664\n", + "T-Statistic: -19.60151487691337\n", + "P-Value: 0.0002900998829147762\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.16899717514124293\n", + "T-Statistic: -3.0669989094316277\n", + "P-Value: 0.054691170931315904\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 665\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 666\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.27329234972677596\n", + "T-Statistic: -2.6358380883941255\n", + "P-Value: 0.07793359764901263\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.21666666666666667\n", + "Average of Other Ratios: 0.15677966101694918\n", + "T-Statistic: -2.8266666666666658\n", + "P-Value: 0.06637194694612157\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 666\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 667\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.2693306010928962\n", + "T-Statistic: -5.264461390477094\n", + "P-Value: 0.013356347106673735\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.17740112994350282\n", + "T-Statistic: -2.5678020379180455\n", + "P-Value: 0.0826509420493295\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 667\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 668\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.22418032786885245\n", + "T-Statistic: -12.077933694787529\n", + "P-Value: 0.0012214555571584704\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.19399717514124293\n", + "T-Statistic: -7.533632446839843\n", + "P-Value: 0.00484812769882256\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 668\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 669\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.19467213114754098\n", + "T-Statistic: -7.753219242030175\n", + "P-Value: 0.004462810515423161\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.21666666666666667\n", + "Average of Other Ratios: 0.1228813559322034\n", + "T-Statistic: -4.990929625938783\n", + "P-Value: 0.015469228914864434\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 669\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 670\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.4375\n", + "T-Statistic: -4.557262825860137\n", + "P-Value: 0.01980483320019867\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.4007768361581921\n", + "T-Statistic: -2.3943057732222077\n", + "P-Value: 0.09636694644723628\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 670\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 671\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8\n", + "Average of Other Ratios: 0.3069672131147541\n", + "T-Statistic: -32.249362363733205\n", + "P-Value: 6.552485880934655e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.3036723163841808\n", + "T-Statistic: -2.439750182371333\n", + "P-Value: 0.09252085884337388\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 671\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 672\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.29836065573770487\n", + "T-Statistic: -4.028754863050068\n", + "P-Value: 0.027487792515242023\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.14058380414312618\n", + "T-Statistic: -2.9676921508875798\n", + "P-Value: 0.0972607614111294\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 672\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 673\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.21666666666666667\n", + "T-Statistic: -24.377049180327866\n", + "P-Value: 0.0001513221713591045\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.18997175141242936\n", + "T-Statistic: -2.727682454452322\n", + "P-Value: 0.07208120831037232\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 673\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 674\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.24870218579234973\n", + "T-Statistic: -9.945483469303717\n", + "P-Value: 0.0021627595309605323\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.17323446327683617\n", + "T-Statistic: -2.443879350950321\n", + "P-Value: 0.09218076354483482\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 674\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 675\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.2364071038251366\n", + "T-Statistic: -7.416787809974502\n", + "P-Value: 0.005071157125568877\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.1606638418079096\n", + "T-Statistic: -2.8273201672963535\n", + "P-Value: 0.06633616259613716\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 675\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 676\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.25\n", + "T-Statistic: -9.035003149610084\n", + "P-Value: 0.0028632366668939696\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.16885593220338985\n", + "T-Statistic: -6.94311791030617\n", + "P-Value: 0.00612758913066015\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 676\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 677\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.25724043715846995\n", + "T-Statistic: -3.574953217198332\n", + "P-Value: 0.03742017108862267\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2833333333333333\n", + "Average of Other Ratios: 0.1694915254237288\n", + "T-Statistic: -8.226203052846838\n", + "P-Value: 0.003760433078739023\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 677\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 678\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.26967213114754096\n", + "T-Statistic: -18.098951116688973\n", + "P-Value: 0.0003679240150022286\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.21511299435028247\n", + "T-Statistic: -3.898316388834917\n", + "P-Value: 0.029954591830056016\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 678\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 679\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.273155737704918\n", + "T-Statistic: -2.9827908163528103\n", + "P-Value: 0.05846639473064487\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.26666666666666666\n", + "Average of Other Ratios: 0.15677966101694915\n", + "T-Statistic: -4.3423725516066725\n", + "P-Value: 0.022543397382713576\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 679\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 680\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3442622950819672\n", + "Average of Other Ratios: 0.20416666666666666\n", + "T-Statistic: -3.023496692892297\n", + "P-Value: 0.0566019195281106\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2\n", + "Average of Other Ratios: 0.14830508474576273\n", + "T-Statistic: -12.19999999999999\n", + "P-Value: 0.001185729108116372\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 680\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 681\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.25293715846994536\n", + "T-Statistic: -3.3702819227387244\n", + "P-Value: 0.04340178717234347\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.12245762711864408\n", + "T-Statistic: -8.865808969091209\n", + "P-Value: 0.0030253402213844243\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 681\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 682\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.19999999999999998\n", + "T-Statistic: -17.24492210110332\n", + "P-Value: 0.00042486852229209587\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.21666666666666667\n", + "Average of Other Ratios: 0.15254237288135594\n", + "T-Statistic: -4.144434018455757\n", + "P-Value: 0.025514895723579813\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 682\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 683\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.2944672131147541\n", + "T-Statistic: -2.8051586461529827\n", + "P-Value: 0.06756322441457491\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.1899011299435028\n", + "T-Statistic: -7.530600502193264\n", + "P-Value: 0.004853749660576098\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 683\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 684\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.14583333333333334\n", + "T-Statistic: -17.025299295981913\n", + "P-Value: 0.00044138624174763324\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.13333333333333333\n", + "Average of Other Ratios: 0.1016949152542373\n", + "T-Statistic: -3.2331615074619027\n", + "P-Value: 0.048100099388329834\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 684\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 685\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.275\n", + "T-Statistic: -2.9340384355396165\n", + "P-Value: 0.060801997551779355\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.1899717514124294\n", + "T-Statistic: -3.1834100610600315\n", + "P-Value: 0.04996302047691425\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 685\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 686\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4098360655737705\n", + "Average of Other Ratios: 0.30833333333333335\n", + "T-Statistic: -5.11676115950563\n", + "P-Value: 0.014446859559659811\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.18566384180790962\n", + "T-Statistic: -4.298438567165345\n", + "P-Value: 0.023162647133957697\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 686\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 687\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.2201502732240437\n", + "T-Statistic: -3.2307527644980745\n", + "P-Value: 0.04818825006053326\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.18333333333333332\n", + "Average of Other Ratios: 0.15677966101694918\n", + "T-Statistic: -6.266666666666664\n", + "P-Value: 0.008201920869688295\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 687\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 688\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4262295081967213\n", + "Average of Other Ratios: 0.2791666666666667\n", + "T-Statistic: -10.333341919958322\n", + "P-Value: 0.0019332884778574828\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.2027542372881356\n", + "T-Statistic: -3.621297736079414\n", + "P-Value: 0.0362143108175595\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 688\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 689\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.21577868852459017\n", + "T-Statistic: -9.111666695534357\n", + "P-Value: 0.0027935663080181916\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.16454802259887005\n", + "T-Statistic: -5.39640733462664\n", + "P-Value: 0.012471369780280921\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 689\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 690\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4262295081967213\n", + "Average of Other Ratios: 0.21250000000000002\n", + "T-Statistic: -8.929327630907842\n", + "P-Value: 0.002963093223014625\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.11807909604519774\n", + "T-Statistic: -10.57181908559856\n", + "P-Value: 0.0018080338565436176\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 690\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 691\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.4375\n", + "T-Statistic: -2.2687169915537657\n", + "P-Value: 0.10805287324168507\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.3463276836158192\n", + "T-Statistic: -6.147522027439089\n", + "P-Value: 0.008659111206240855\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 691\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 692\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.2916666666666667\n", + "T-Statistic: -6.359769644709016\n", + "P-Value: 0.007866504158921796\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.2615819209039548\n", + "T-Statistic: -4.063904567563298\n", + "P-Value: 0.026868139751298294\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 692\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 693\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.39344262295081966\n", + "Average of Other Ratios: 0.26666666666666666\n", + "T-Statistic: -1.8012412407306406\n", + "P-Value: 0.1694691787567682\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.1644774011299435\n", + "T-Statistic: -5.495043416350755\n", + "P-Value: 0.011859301638986507\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 693\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 694\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.49166666666666675\n", + "T-Statistic: -6.916091828453289\n", + "P-Value: 0.006196281549258644\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.45593220338983054\n", + "T-Statistic: -4.34666107141259\n", + "P-Value: 0.02248410342384351\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 694\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 695\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.2776639344262295\n", + "T-Statistic: -1.7553554396997046\n", + "P-Value: 0.1774643858274755\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.2068502824858757\n", + "T-Statistic: -3.4595325465113227\n", + "P-Value: 0.04065396019819478\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 695\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 696\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36065573770491804\n", + "Average of Other Ratios: 0.2916666666666667\n", + "T-Statistic: -1.899261403354651\n", + "P-Value: 0.15374343061615317\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.17711864406779662\n", + "T-Statistic: -8.161598246688756\n", + "P-Value: 0.0038473182454779415\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 696\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 697\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.26584699453551913\n", + "T-Statistic: -4.361077099679005\n", + "P-Value: 0.02228625751121354\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.1730225988700565\n", + "T-Statistic: -6.742490376010055\n", + "P-Value: 0.006662617945926082\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 697\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 698\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4262295081967213\n", + "Average of Other Ratios: 0.27083333333333337\n", + "T-Statistic: -3.735739626625215\n", + "P-Value: 0.03344241564305693\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.15176553672316384\n", + "T-Statistic: -5.775098949219386\n", + "P-Value: 0.010322711204428422\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 698\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 699\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.2375\n", + "T-Statistic: -7.082180009413329\n", + "P-Value: 0.0057895528079337285\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.1434322033898305\n", + "T-Statistic: -13.161604379732514\n", + "P-Value: 0.0009475271511557978\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 699\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 700\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4262295081967213\n", + "Average of Other Ratios: 0.25416666666666665\n", + "T-Statistic: -4.377269933993477\n", + "P-Value: 0.02206670433265803\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.18121468926553674\n", + "T-Statistic: -2.37554254984018\n", + "P-Value: 0.0980116289989706\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 700\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 701\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.24583333333333332\n", + "T-Statistic: -5.194910461282009\n", + "P-Value: 0.013855966641037953\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.21666666666666667\n", + "Average of Other Ratios: 0.1440677966101695\n", + "T-Statistic: -4.11529355279074\n", + "P-Value: 0.025994211990808435\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 701\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 702\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.28169398907103826\n", + "T-Statistic: -5.477980736046312\n", + "P-Value: 0.011962325656887837\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.19837570621468928\n", + "T-Statistic: -3.4094802718971997\n", + "P-Value: 0.042166766315337265\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 702\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 703\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.2239754098360656\n", + "T-Statistic: -10.180119908482158\n", + "P-Value: 0.0020198909534647554\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.1731638418079096\n", + "T-Statistic: -7.679245283018867\n", + "P-Value: 0.004587946327225194\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 703\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 704\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.2125\n", + "T-Statistic: -4.076969624956957\n", + "P-Value: 0.0266424079833841\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.21666666666666667\n", + "Average of Other Ratios: 0.1440677966101695\n", + "T-Statistic: -4.945967306057795\n", + "P-Value: 0.015857343159367175\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 704\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 705\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3689890710382514\n", + "T-Statistic: -4.353209549753575\n", + "P-Value: 0.022393951473013263\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3418079096045198\n", + "T-Statistic: -5.938574464184704\n", + "P-Value: 0.00954502881900154\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 705\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 706\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4098360655737705\n", + "Average of Other Ratios: 0.23333333333333334\n", + "T-Statistic: -3.01552163240712\n", + "P-Value: 0.05696125979179388\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.13509887005649718\n", + "T-Statistic: -4.199506762976739\n", + "P-Value: 0.02463973970623848\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 706\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 707\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.20416666666666666\n", + "T-Statistic: -18.1470510379736\n", + "P-Value: 0.0003650272002577954\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.1346045197740113\n", + "T-Statistic: -2.8897637795275584\n", + "P-Value: 0.21209033041734834\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 707\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 708\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.2125\n", + "T-Statistic: -8.356089793398965\n", + "P-Value: 0.003593448152596275\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.14343220338983054\n", + "T-Statistic: -4.115302637249344\n", + "P-Value: 0.025994060763921405\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 708\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 709\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.4937841530054645\n", + "T-Statistic: -13.399578464312397\n", + "P-Value: 0.000898581498174531\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5932203389830508\n", + "Average of Other Ratios: 0.4980225988700565\n", + "T-Statistic: -2.5424731748068985\n", + "P-Value: 0.08449660749382112\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 709\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 710\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.22916666666666669\n", + "T-Statistic: -5.559506115611259\n", + "P-Value: 0.01148039203424712\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.17747175141242938\n", + "T-Statistic: -4.301532117285934\n", + "P-Value: 0.023118328990842697\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 710\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 711\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.2409153005464481\n", + "T-Statistic: -5.275556807499205\n", + "P-Value: 0.013278830036287586\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3333333333333333\n", + "Average of Other Ratios: 0.2076271186440678\n", + "T-Statistic: -3.6796971261465163\n", + "P-Value: 0.03476456233229096\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 711\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 712\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.39344262295081966\n", + "Average of Other Ratios: 0.25\n", + "T-Statistic: -4.714005617872331\n", + "P-Value: 0.018074445236289073\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.17309322033898306\n", + "T-Statistic: -2.2132516655917454\n", + "P-Value: 0.11375613879223859\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 712\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 713\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.3041666666666667\n", + "T-Statistic: -6.34727085863382\n", + "P-Value: 0.007910472276561024\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.17323446327683617\n", + "T-Statistic: -3.2772900040790653\n", + "P-Value: 0.046520687780565555\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 713\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 714\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.2657103825136612\n", + "T-Statistic: -3.2393534601694043\n", + "P-Value: 0.04787443221098791\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.18149717514124292\n", + "T-Statistic: -14.714285714285706\n", + "P-Value: 0.0006808928260169268\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 714\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 715\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4649590163934426\n", + "T-Statistic: -6.9498273973418225\n", + "P-Value: 0.0061106911480797435\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.417725988700565\n", + "T-Statistic: -10.18365359023622\n", + "P-Value: 0.0020178363490044415\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 715\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 716\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8360655737704918\n", + "Average of Other Ratios: 0.4375\n", + "T-Statistic: -8.601708046331813\n", + "P-Value: 0.0033035386244482587\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4609227871939736\n", + "T-Statistic: -2.429062741718821\n", + "P-Value: 0.13579717414652953\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 716\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 717\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.2448087431693989\n", + "T-Statistic: -8.253674606606968\n", + "P-Value: 0.0037242731002333166\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.2024717514124294\n", + "T-Statistic: -4.532253548113998\n", + "P-Value: 0.020100540272823354\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 717\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 718\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.24057377049180328\n", + "T-Statistic: -5.639493439517238\n", + "P-Value: 0.011031947405375732\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.14774011299435028\n", + "T-Statistic: -3.52461629327718\n", + "P-Value: 0.03878859478065401\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 718\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 719\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.21530054644808744\n", + "T-Statistic: -9.600064444773704\n", + "P-Value: 0.002398495294738282\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.16101694915254236\n", + "T-Statistic: -6.609891577527327\n", + "P-Value: 0.007050376214881156\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 719\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 720\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.26140710382513666\n", + "T-Statistic: -8.577234238877079\n", + "P-Value: 0.003331005620411765\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.21525423728813559\n", + "T-Statistic: -6.513721780101715\n", + "P-Value: 0.007350361959320121\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 720\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 721\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.43333333333333335\n", + "T-Statistic: -18.728885297038175\n", + "P-Value: 0.00033227343416865454\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4175847457627119\n", + "T-Statistic: -8.4047123277365\n", + "P-Value: 0.0035334622322826825\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 721\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 722\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.27827868852459015\n", + "T-Statistic: -3.3172720549176096\n", + "P-Value: 0.04514599422781829\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3\n", + "Average of Other Ratios: 0.1652542372881356\n", + "T-Statistic: -6.045729350003519\n", + "P-Value: 0.009076571184230395\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 722\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 723\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.2375\n", + "T-Statistic: -10.122383386195422\n", + "P-Value: 0.0020538573428606206\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.16440677966101697\n", + "T-Statistic: -2.6381526558147996\n", + "P-Value: 0.07777904054371029\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 723\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 724\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.3\n", + "T-Statistic: -3.5639741636483593\n", + "P-Value: 0.037713321483800306\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3\n", + "Average of Other Ratios: 0.19491525423728814\n", + "T-Statistic: -3.201666232864797\n", + "P-Value: 0.04926900790572485\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 724\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 725\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.24480874316939888\n", + "T-Statistic: -2.9309984722316638\n", + "P-Value: 0.06095147293868425\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.15176553672316384\n", + "T-Statistic: -3.080747359192189\n", + "P-Value: 0.05410421984381787\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 725\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 726\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.31912568306010924\n", + "T-Statistic: -2.530970883389505\n", + "P-Value: 0.08535147636509026\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.19823446327683616\n", + "T-Statistic: -7.231156579885582\n", + "P-Value: 0.005454152940167432\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 726\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 727\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.24453551912568305\n", + "T-Statistic: -6.751934642062348\n", + "P-Value: 0.006636082160327463\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.26666666666666666\n", + "Average of Other Ratios: 0.18220338983050846\n", + "T-Statistic: -4.8345434590575715\n", + "P-Value: 0.016875322539984233\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 727\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 728\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3333333333333333\n", + "Average of Other Ratios: 0.24890710382513662\n", + "T-Statistic: -2.3788710766712677\n", + "P-Value: 0.14042438082786876\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.15621468926553672\n", + "T-Statistic: -6.014126755966144\n", + "P-Value: 0.009211533394928783\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 728\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 729\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4769125683060109\n", + "T-Statistic: -3.385054287067082\n", + "P-Value: 0.04293102665536962\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4514124293785311\n", + "T-Statistic: -2.773457470011942\n", + "P-Value: 0.06936804083315011\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 729\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 730\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.24904371584699453\n", + "T-Statistic: -16.87653237880174\n", + "P-Value: 0.00045306323873643635\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.1942090395480226\n", + "T-Statistic: -6.695816773490067\n", + "P-Value: 0.006795828785145915\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 730\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 731\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.25416666666666665\n", + "T-Statistic: -4.94476472119911\n", + "P-Value: 0.01586789705539504\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.19823446327683614\n", + "T-Statistic: -4.580754969794434\n", + "P-Value: 0.019532183920823603\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 731\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 732\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.39344262295081966\n", + "Average of Other Ratios: 0.17916666666666667\n", + "T-Statistic: -7.782330369943198\n", + "P-Value: 0.004414801539065831\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2\n", + "Average of Other Ratios: 0.11440677966101695\n", + "T-Statistic: -4.899219625733926\n", + "P-Value: 0.016274424287796678\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 732\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 733\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.29829234972677593\n", + "T-Statistic: -3.6409046416348776\n", + "P-Value: 0.035719102507377344\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.21504237288135594\n", + "T-Statistic: -2.93071839321159\n", + "P-Value: 0.060965267594478456\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 733\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 734\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.18333333333333335\n", + "T-Statistic: -31.10048861943559\n", + "P-Value: 7.303903972764834e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.1772598870056497\n", + "T-Statistic: -4.332361052668546\n", + "P-Value: 0.022682607763125775\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 734\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 735\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.2\n", + "T-Statistic: -9.42090076205074\n", + "P-Value: 0.0025342671020132644\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.1348870056497175\n", + "T-Statistic: -2.4242425425396985\n", + "P-Value: 0.09381189733415801\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 735\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 736\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.22499999999999998\n", + "T-Statistic: -5.482448602285279\n", + "P-Value: 0.011935236042507276\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.17747175141242938\n", + "T-Statistic: -2.3175577935580045\n", + "P-Value: 0.10331445267128257\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 736\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 737\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.5806010928961749\n", + "T-Statistic: -3.3948578066343797\n", + "P-Value: 0.04262218843494368\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5932203389830508\n", + "Average of Other Ratios: 0.5064971751412429\n", + "T-Statistic: -3.4740867918071037\n", + "P-Value: 0.04022706931462362\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 737\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 738\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.29166666666666663\n", + "T-Statistic: -8.21042730655783\n", + "P-Value: 0.0037814081546606237\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.2542372881355932\n", + "T-Statistic: -5.230883563714916\n", + "P-Value: 0.013594564408150057\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 738\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 739\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.32786885245901637\n", + "Average of Other Ratios: 0.1958333333333333\n", + "T-Statistic: -5.118158442680822\n", + "P-Value: 0.014436008902853572\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.15254237288135594\n", + "Average of Other Ratios: 0.11377118644067798\n", + "T-Statistic: -2.1108984192580493\n", + "P-Value: 0.12526011393630182\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 739\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 740\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.23210382513661199\n", + "T-Statistic: -12.123135549004308\n", + "P-Value: 0.0012080613029538494\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.16031073446327682\n", + "T-Statistic: -2.958934625443265\n", + "P-Value: 0.05959496241150818\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 740\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 741\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.2116120218579235\n", + "T-Statistic: -23.085068346256453\n", + "P-Value: 0.00017805380137088774\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.1480225988700565\n", + "T-Statistic: -3.1535457618776688\n", + "P-Value: 0.05112515625755018\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 741\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 742\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4262295081967213\n", + "Average of Other Ratios: 0.3\n", + "T-Statistic: -2.9706771945521644\n", + "P-Value: 0.05903607413760807\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.1812853107344633\n", + "T-Statistic: -7.814244577012594\n", + "P-Value: 0.004362950284160135\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 742\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 743\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4262295081967213\n", + "Average of Other Ratios: 0.23333333333333334\n", + "T-Statistic: -2.8349832104998747\n", + "P-Value: 0.0659183394825102\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.12655367231638417\n", + "T-Statistic: -9.201899164855353\n", + "P-Value: 0.0027144108276950137\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 743\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 744\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.2941256830601093\n", + "T-Statistic: -5.153821208882444\n", + "P-Value: 0.014162630156722995\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.19830508474576272\n", + "T-Statistic: -5.284229075567873\n", + "P-Value: 0.01321865065791954\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 744\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 745\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4262295081967213\n", + "Average of Other Ratios: 0.24583333333333335\n", + "T-Statistic: -2.8527267194908648\n", + "P-Value: 0.06496341216084892\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.1475988700564972\n", + "T-Statistic: -7.72844712666813\n", + "P-Value: 0.00450420801537689\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 745\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 746\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.2069672131147541\n", + "T-Statistic: -3.211725828402651\n", + "P-Value: 0.048891793073391886\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.18333333333333332\n", + "Average of Other Ratios: 0.1440677966101695\n", + "T-Statistic: -3.5889645674855384\n", + "P-Value: 0.037050266207125974\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 746\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 747\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.2360655737704918\n", + "T-Statistic: -3.510803607398461\n", + "P-Value: 0.03917521433599371\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.13919491525423727\n", + "T-Statistic: -4.348997984611037\n", + "P-Value: 0.022451877691097657\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 747\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 748\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.3026639344262295\n", + "T-Statistic: -2.8694731663260846\n", + "P-Value: 0.06407793061054502\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.2076271186440678\n", + "T-Statistic: -6.0666666666666735\n", + "P-Value: 0.008988582402196459\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 748\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 749\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.21666666666666667\n", + "T-Statistic: -7.61224281479834\n", + "P-Value: 0.004705316919230158\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.16857344632768362\n", + "T-Statistic: -3.433022580597172\n", + "P-Value: 0.04144644945716894\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 749\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 750\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.2573087431693989\n", + "T-Statistic: -5.082825548341707\n", + "P-Value: 0.014713685416969534\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.21666666666666667\n", + "Average of Other Ratios: 0.1694915254237288\n", + "T-Statistic: -2.5768659443669355\n", + "P-Value: 0.08200254809923566\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 750\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 751\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.22862021857923498\n", + "T-Statistic: -3.065403117058221\n", + "P-Value: 0.05475981670342542\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.12217514124293785\n", + "T-Statistic: -6.391226809216671\n", + "P-Value: 0.0077572568493554745\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 751\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 752\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.26509562841530054\n", + "T-Statistic: -2.866308813315449\n", + "P-Value: 0.06424408813835332\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.15204802259887007\n", + "T-Statistic: -4.146966401618811\n", + "P-Value: 0.025473782147724618\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 752\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 753\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.30273224043715846\n", + "T-Statistic: -5.035807306036759\n", + "P-Value: 0.015094095073797963\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1856638418079096\n", + "T-Statistic: -7.414159169988326\n", + "P-Value: 0.005076328764679312\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 753\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 754\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.3404371584699453\n", + "T-Statistic: -6.119691979185821\n", + "P-Value: 0.008770696803910683\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.2827683615819209\n", + "T-Statistic: -2.946612591077136\n", + "P-Value: 0.06018857902224991\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 754\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 755\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.20416666666666666\n", + "T-Statistic: -7.431074551672135\n", + "P-Value: 0.005043170304209105\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.16857344632768362\n", + "T-Statistic: -3.9354028393368354\n", + "P-Value: 0.02922522554345655\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 755\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 756\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.25\n", + "T-Statistic: -6.605727113313177\n", + "P-Value: 0.007063029498989654\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.1855225988700565\n", + "T-Statistic: -9.224885508045242\n", + "P-Value: 0.002694722054185784\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 756\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 757\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.2698087431693989\n", + "T-Statistic: -9.036379043539897\n", + "P-Value: 0.0028619661292264213\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.18149717514124294\n", + "T-Statistic: -5.610767520239283\n", + "P-Value: 0.011190314670533465\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 757\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 758\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.2791666666666667\n", + "T-Statistic: -7.116030982441239\n", + "P-Value: 0.005711005406184151\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.21532485875706214\n", + "T-Statistic: -4.672257354206355\n", + "P-Value: 0.018515494454469265\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 758\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 759\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.28620218579234974\n", + "T-Statistic: -3.720511834007275\n", + "P-Value: 0.033795204522967935\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.19413841807909604\n", + "T-Statistic: -14.324297025025746\n", + "P-Value: 0.000737365952799921\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 759\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 760\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.2201502732240437\n", + "T-Statistic: -5.295186309462342\n", + "P-Value: 0.013143123507478411\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.15\n", + "Average of Other Ratios: 0.1271186440677966\n", + "T-Statistic: -4.67653718043597\n", + "P-Value: 0.018469641646674036\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 760\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 761\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.28217213114754097\n", + "T-Statistic: -16.292961239551758\n", + "P-Value: 0.0005030512239375957\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.23022598870056496\n", + "T-Statistic: -2.4285714285714284\n", + "P-Value: 0.13584143478190688\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 761\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 762\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.275\n", + "T-Statistic: -3.8509508464272586\n", + "P-Value: 0.03092048438850596\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.1571563088512241\n", + "T-Statistic: -6.947685432148197\n", + "P-Value: 0.020094339591082675\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 762\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 763\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3770491803278688\n", + "Average of Other Ratios: 0.24166666666666664\n", + "T-Statistic: -6.140373651674497\n", + "P-Value: 0.008687594169327536\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.12919020715630886\n", + "T-Statistic: -5.005419915905\n", + "P-Value: 0.03767240357329794\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 763\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 764\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.17397540983606558\n", + "T-Statistic: -18.4422633103658\n", + "P-Value: 0.0003478966667273698\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1694915254237288\n", + "Average of Other Ratios: 0.13072033898305085\n", + "T-Statistic: -2.4581295560193186\n", + "P-Value: 0.09101873468865801\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 764\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 765\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.4228142076502732\n", + "T-Statistic: -4.415045976602256\n", + "P-Value: 0.02156528848562188\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.4389830508474576\n", + "T-Statistic: -4.148687032732543\n", + "P-Value: 0.025445896223527123\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 765\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 766\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.19999999999999998\n", + "T-Statistic: -8.806809104164614\n", + "P-Value: 0.003084718357015228\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.14745762711864407\n", + "T-Statistic: -3.1135770454260725\n", + "P-Value: 0.052734401788326495\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 766\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 767\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.20314207650273225\n", + "T-Statistic: -9.794313683048983\n", + "P-Value: 0.002261966026846209\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.1432909604519774\n", + "T-Statistic: -3.097057119520469\n", + "P-Value: 0.053418165120319126\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 767\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 768\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.2532103825136612\n", + "T-Statistic: -5.1768582175628834\n", + "P-Value: 0.013989618848905717\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.15197740112994348\n", + "T-Statistic: -5.438132165795431\n", + "P-Value: 0.012207529595140413\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 768\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 769\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.2275273224043716\n", + "T-Statistic: -5.726067308431419\n", + "P-Value: 0.010572065566569503\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.16871468926553673\n", + "T-Statistic: -5.51623494926466\n", + "P-Value: 0.011732957333623041\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 769\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 770\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.1708333333333333\n", + "T-Statistic: -12.10156988574371\n", + "P-Value: 0.0012144271996402047\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.15677966101694915\n", + "T-Statistic: -2.5809523809523816\n", + "P-Value: 0.08171227084668078\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 770\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 771\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4098360655737705\n", + "Average of Other Ratios: 0.23333333333333334\n", + "T-Statistic: -5.951158369984666\n", + "P-Value: 0.00948839549552589\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1393361581920904\n", + "T-Statistic: -11.45967949880383\n", + "P-Value: 0.001426179984263791\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 771\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 772\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.20416666666666666\n", + "T-Statistic: -4.968520257961496\n", + "P-Value: 0.015661100563564314\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.21666666666666667\n", + "Average of Other Ratios: 0.13983050847457626\n", + "T-Statistic: -3.156608161329988\n", + "P-Value: 0.05100442714599591\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 772\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 773\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.2527322404371585\n", + "T-Statistic: -4.948933058520406\n", + "P-Value: 0.01583135472899473\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.20268361581920905\n", + "T-Statistic: -2.734913303711143\n", + "P-Value: 0.07164396808065115\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 773\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 774\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.25\n", + "T-Statistic: -4.9384820758662515\n", + "P-Value: 0.015923182580957953\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.16318267419962337\n", + "T-Statistic: -2.2651619180423666\n", + "P-Value: 0.15174721013539427\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 774\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 775\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.2739071038251366\n", + "T-Statistic: -13.930528616417067\n", + "P-Value: 0.0008008837712144284\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.1814265536723164\n", + "T-Statistic: -3.4427734423525815\n", + "P-Value: 0.04115269085326815\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 775\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 776\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.15833333333333333\n", + "T-Statistic: -7.384810731767565\n", + "P-Value: 0.005134544759237181\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2033898305084746\n", + "Average of Other Ratios: 0.16031073446327682\n", + "T-Statistic: -5.211581712072246\n", + "P-Value: 0.01373401507365159\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 776\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 777\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.30273224043715846\n", + "T-Statistic: -4.0696430409706865\n", + "P-Value: 0.026768690892997455\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.2573446327683616\n", + "T-Statistic: -5.309251443565037\n", + "P-Value: 0.01304699781574515\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 777\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 778\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3855191256830601\n", + "T-Statistic: -2.636692552502379\n", + "P-Value: 0.07787649597226969\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.33709981167608283\n", + "T-Statistic: -28.00000000000001\n", + "P-Value: 0.0012730749910096228\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 778\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 779\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.22083333333333333\n", + "T-Statistic: -8.148412107258412\n", + "P-Value: 0.003865377073276304\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.14759887005649716\n", + "T-Statistic: -7.540447009277395\n", + "P-Value: 0.00483552316841547\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 779\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 780\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.298224043715847\n", + "T-Statistic: -2.2807020090731873\n", + "P-Value: 0.10686619566510568\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.1561440677966102\n", + "T-Statistic: -14.299454098945166\n", + "P-Value: 0.0007411711769066464\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 780\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 781\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4262295081967213\n", + "Average of Other Ratios: 0.18333333333333332\n", + "T-Statistic: -4.17817014100824\n", + "P-Value: 0.02497413039088068\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.13481638418079095\n", + "T-Statistic: -4.017937620623259\n", + "P-Value: 0.027682186108019237\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 781\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 782\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.27322404371584696\n", + "T-Statistic: -5.040542454345847\n", + "P-Value: 0.015055208208642631\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.1426553672316384\n", + "T-Statistic: -3.941176470588237\n", + "P-Value: 0.15819200517512308\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 782\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 783\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.21530054644808744\n", + "T-Statistic: -4.283002068724982\n", + "P-Value: 0.02338543904381472\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.15254237288135594\n", + "Average of Other Ratios: 0.10988700564971751\n", + "T-Statistic: -2.589247891896882\n", + "P-Value: 0.08112688802619472\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 783\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 784\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.19583333333333333\n", + "T-Statistic: -7.410760977286471\n", + "P-Value: 0.005083024752688326\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.21666666666666667\n", + "Average of Other Ratios: 0.13559322033898305\n", + "T-Statistic: -4.428506143912038\n", + "P-Value: 0.021390194677135307\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 784\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 785\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3442622950819672\n", + "Average of Other Ratios: 0.27499999999999997\n", + "T-Statistic: -4.798632565231811\n", + "P-Value: 0.017221458650986843\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.16864406779661018\n", + "T-Statistic: -3.4776799529026854\n", + "P-Value: 0.040122557696007824\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 785\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 786\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.2864071038251366\n", + "T-Statistic: -4.100734975708682\n", + "P-Value: 0.02623802994170771\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.17309322033898306\n", + "T-Statistic: -7.501973136790743\n", + "P-Value: 0.004907258242497694\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 786\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 787\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.31666666666666665\n", + "T-Statistic: -4.974054989029201\n", + "P-Value: 0.015613424561556911\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.24449152542372882\n", + "T-Statistic: -3.749896613676689\n", + "P-Value: 0.03311865459697996\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 787\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 788\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.19863387978142077\n", + "T-Statistic: -6.300594478470427\n", + "P-Value: 0.00807755369838508\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.1271186440677966\n", + "T-Statistic: -7.236123373843309\n", + "P-Value: 0.005443417574107074\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 788\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 789\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.244603825136612\n", + "T-Statistic: -2.2254146723514387\n", + "P-Value: 0.11247501852404906\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.14613935969868172\n", + "T-Statistic: -3.446237951244447\n", + "P-Value: 0.07486648577768765\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 789\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 790\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.2041666666666667\n", + "T-Statistic: -13.312635870763579\n", + "P-Value: 0.0009160652152262239\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.16871468926553673\n", + "T-Statistic: -4.7295667427478545\n", + "P-Value: 0.017913544696107814\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 790\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 791\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.4937158469945355\n", + "T-Statistic: -8.885516696979227\n", + "P-Value: 0.003005843404969766\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4600988700564972\n", + "T-Statistic: -2.332818394140734\n", + "P-Value: 0.10188575953545359\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 791\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 792\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.24166666666666664\n", + "T-Statistic: -13.62086314551715\n", + "P-Value: 0.0008560385924757967\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.22033898305084745\n", + "Average of Other Ratios: 0.1814265536723164\n", + "T-Statistic: -4.854861096609305\n", + "P-Value: 0.016683488889815655\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 792\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 793\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.26154371584699454\n", + "T-Statistic: -3.4496293922245655\n", + "P-Value: 0.040947730245819165\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.1350282485875706\n", + "T-Statistic: -5.941894164731409\n", + "P-Value: 0.027174489715421215\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 793\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 794\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.29890710382513663\n", + "T-Statistic: -3.8375524808634864\n", + "P-Value: 0.031200942303648546\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.21518361581920906\n", + "T-Statistic: -2.4427691887863823\n", + "P-Value: 0.09227205070358749\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 794\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 795\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.2693306010928962\n", + "T-Statistic: -6.246793566459261\n", + "P-Value: 0.008275938325843453\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.21610169491525422\n", + "T-Statistic: -9.572072668100612\n", + "P-Value: 0.002419061759112068\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 795\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 796\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.2655054644808743\n", + "T-Statistic: -9.468017765185019\n", + "P-Value: 0.002497594914513529\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.19823446327683616\n", + "T-Statistic: -5.967112085495144\n", + "P-Value: 0.009417228305284209\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 796\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 797\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.19501366120218577\n", + "T-Statistic: -6.412883435087076\n", + "P-Value: 0.007683202378919043\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.15254237288135594\n", + "Average of Other Ratios: 0.12238700564971752\n", + "T-Statistic: -6.812191696797892\n", + "P-Value: 0.00647001003369456\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 797\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 798\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.30683060109289617\n", + "T-Statistic: -2.6055084852992985\n", + "P-Value: 0.07999436123101161\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.22379943502824862\n", + "T-Statistic: -6.706721318121227\n", + "P-Value: 0.006764394593913462\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 798\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 799\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.19999999999999998\n", + "T-Statistic: -6.752251856457131\n", + "P-Value: 0.006635193291969306\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.17281073446327685\n", + "T-Statistic: -5.072284754402177\n", + "P-Value: 0.014797870994727638\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 799\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 800\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.20833333333333334\n", + "T-Statistic: -19.77027395258561\n", + "P-Value: 0.0002827789468772716\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.16299435028248588\n", + "T-Statistic: -3.608695652173914\n", + "P-Value: 0.06894253641177729\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 800\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 801\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4208333333333334\n", + "T-Statistic: -10.236282712367263\n", + "P-Value: 0.0019875612473827316\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.38411016949152543\n", + "T-Statistic: -7.205444253314168\n", + "P-Value: 0.005510176845082696\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 801\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 802\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.4541666666666667\n", + "T-Statistic: -4.489735890023653\n", + "P-Value: 0.020616535562147837\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.36563088512241054\n", + "T-Statistic: -2.235697940503432\n", + "P-Value: 0.1548857125971983\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 802\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 803\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.36038251366120216\n", + "T-Statistic: -4.0670388186695\n", + "P-Value: 0.02681376369186907\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.36864406779661013\n", + "T-Statistic: -3.21491849579106\n", + "P-Value: 0.04877283804441102\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 803\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 804\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.4560792349726776\n", + "T-Statistic: -5.9382601309311465\n", + "P-Value: 0.009546449129946996\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4011299435028249\n", + "T-Statistic: -4.050528318752622\n", + "P-Value: 0.027101807358158075\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 804\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 805\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.425\n", + "T-Statistic: -2.2720572719623497\n", + "P-Value: 0.10772054769411174\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.36737288135593216\n", + "T-Statistic: -3.6999746626775307\n", + "P-Value: 0.03427858129419643\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 805\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 806\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.45\n", + "T-Statistic: -3.6218924086243263\n", + "P-Value: 0.036199162722362586\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.4135593220338983\n", + "T-Statistic: -4.495611895592143\n", + "P-Value: 0.02054420972468061\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 806\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 807\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.38545081967213113\n", + "T-Statistic: -5.020124422686412\n", + "P-Value: 0.015223827432133234\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3290960451977401\n", + "T-Statistic: -3.025290226140455\n", + "P-Value: 0.05652149905978702\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 807\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 808\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4937158469945355\n", + "T-Statistic: -3.4194195863846306\n", + "P-Value: 0.041860729913222816\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.4639830508474576\n", + "T-Statistic: -2.3727172680373334\n", + "P-Value: 0.09826222788434269\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 808\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 809\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.5\n", + "T-Statistic: -2.260714065293725\n", + "P-Value: 0.10885413598684734\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.43898305084745765\n", + "T-Statistic: -3.136812146700215\n", + "P-Value: 0.051791252064485205\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 809\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 810\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.3735655737704918\n", + "T-Statistic: -10.454060628342173\n", + "P-Value: 0.0018684937780359754\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.35169491525423724\n", + "T-Statistic: -2.252629691622204\n", + "P-Value: 0.10967084489225898\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 810\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 811\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -9.756144412236056\n", + "P-Value: 0.0022879647959347044\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.4091101694915254\n", + "T-Statistic: -2.4465756114105797\n", + "P-Value: 0.09195951277963867\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 811\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 812\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.39583333333333337\n", + "T-Statistic: -7.4590163934426235\n", + "P-Value: 0.004989020225851709\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.4260593220338983\n", + "T-Statistic: -2.3835678188830465\n", + "P-Value: 0.0973040279791015\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 812\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 813\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.3608606557377049\n", + "T-Statistic: -12.95979435165334\n", + "P-Value: 0.0009918481137665518\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.32902542372881355\n", + "T-Statistic: -6.272727272727274\n", + "P-Value: 0.0081795212436495\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 813\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 814\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -10.272001058557553\n", + "P-Value: 0.001967356422200276\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3584039548022599\n", + "T-Statistic: -2.8942424583125095\n", + "P-Value: 0.06279568913296162\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 814\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 815\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.435655737704918\n", + "T-Statistic: -6.464633818447615\n", + "P-Value: 0.00750996829937376\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.3962570621468927\n", + "T-Statistic: -4.31678224360102\n", + "P-Value: 0.022901451094550856\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 815\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 816\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.45211748633879784\n", + "T-Statistic: -15.725247417039386\n", + "P-Value: 0.0005589740150536343\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.438771186440678\n", + "T-Statistic: -3.9134326914916078\n", + "P-Value: 0.029654511991928277\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 816\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 817\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.4125\n", + "T-Statistic: -3.7159734017373043\n", + "P-Value: 0.033901269284630456\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3586158192090395\n", + "T-Statistic: -11.057143071059498\n", + "P-Value: 0.0015845269383183932\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 817\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 818\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.40191256830601096\n", + "T-Statistic: -2.7474782254251675\n", + "P-Value: 0.07089198741296619\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3093220338983051\n", + "T-Statistic: -2.0349703424076058\n", + "P-Value: 0.1788266478229615\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 818\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 819\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.45642076502732243\n", + "T-Statistic: -10.118093819799428\n", + "P-Value: 0.0020564110187835205\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4639124293785311\n", + "T-Statistic: -3.1846701192429343\n", + "P-Value: 0.049914723854652635\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 819\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 820\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.39822404371584696\n", + "T-Statistic: -2.815880143112684\n", + "P-Value: 0.06696608581805782\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.40091807909604515\n", + "T-Statistic: -2.9501131862499728\n", + "P-Value: 0.060019186645291704\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 820\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 821\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.48333333333333334\n", + "T-Statistic: -5.404377109955353\n", + "P-Value: 0.012420402409580856\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.41572504708097924\n", + "T-Statistic: -2.4051807590126426\n", + "P-Value: 0.13797203364885854\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 821\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 822\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -6.759987541853552\n", + "P-Value: 0.006613565506268803\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3145009416195857\n", + "T-Statistic: -9.800000000000015\n", + "P-Value: 0.010252475022698292\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 822\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 823\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.4916666666666667\n", + "T-Statistic: -6.965216353380622\n", + "P-Value: 0.006072164935164103\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.42664783427495295\n", + "T-Statistic: -3.434253416143983\n", + "P-Value: 0.07533225481738264\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 823\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 824\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3817622950819672\n", + "T-Statistic: -6.45646486744287\n", + "P-Value: 0.007536970061691775\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.35847457627118645\n", + "T-Statistic: -3.7468218576747336\n", + "P-Value: 0.03318862977572546\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 824\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 825\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.425\n", + "T-Statistic: -18.39984547099852\n", + "P-Value: 0.0003502913010235412\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.4152542372881356\n", + "T-Statistic: -2.677777777777778\n", + "P-Value: 0.07519088494134536\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 825\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 826\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.3583333333333334\n", + "T-Statistic: -6.037002655406131\n", + "P-Value: 0.009113578699967435\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3415960451977401\n", + "T-Statistic: -2.404160372045395\n", + "P-Value: 0.09551661130031602\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 826\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 827\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -8.311288381099306\n", + "P-Value: 0.003649917396478642\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3686440677966102\n", + "T-Statistic: -3.7363704002836675\n", + "P-Value: 0.03342790419303777\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 827\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 828\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.3833333333333333\n", + "T-Statistic: -5.370455913263936\n", + "P-Value: 0.012639237313128257\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3855932203389831\n", + "T-Statistic: -3.427505887692899\n", + "P-Value: 0.041613829518252106\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 828\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 829\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.44371584699453553\n", + "T-Statistic: -2.834044674981603\n", + "P-Value: 0.06596933591930301\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.45572033898305087\n", + "T-Statistic: -2.7857691081092364\n", + "P-Value: 0.06866006993419116\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 829\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 830\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.5101092896174864\n", + "T-Statistic: -3.547608678177134\n", + "P-Value: 0.038155756341829127\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4322033898305085\n", + "T-Statistic: -5.013002700820275\n", + "P-Value: 0.015283219713031168\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 830\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 831\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.43558743169398906\n", + "T-Statistic: -3.3200966453052976\n", + "P-Value: 0.045050836488169606\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.41786723163841805\n", + "T-Statistic: -2.9961510064713686\n", + "P-Value: 0.057846064903379533\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 831\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 832\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4814890710382514\n", + "T-Statistic: -8.921688593482292\n", + "P-Value: 0.0029704891181838475\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.576271186440678\n", + "Average of Other Ratios: 0.384180790960452\n", + "T-Statistic: -4.605541138054455\n", + "P-Value: 0.01924977204112997\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 832\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 833\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.36249999999999993\n", + "T-Statistic: -10.885700407196474\n", + "P-Value: 0.0016590620208720404\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4048022598870057\n", + "T-Statistic: -3.1550201885686455\n", + "P-Value: 0.051066984905793825\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 833\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 834\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.4625\n", + "T-Statistic: -3.8807823860207193\n", + "P-Value: 0.030307571526173836\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3936911487758945\n", + "T-Statistic: -1.6469089297144321\n", + "P-Value: 0.24133036369239244\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 834\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 835\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -3.796834767979554\n", + "P-Value: 0.032073534237681285\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.38983050847457623\n", + "T-Statistic: -6.0432055511403355\n", + "P-Value: 0.009087253623111777\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 835\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 836\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.35\n", + "T-Statistic: -6.095615458270268\n", + "P-Value: 0.008868760073876481\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3036723163841808\n", + "T-Statistic: -4.41816098848226\n", + "P-Value: 0.021524602925384974\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 836\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 837\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4852459016393443\n", + "T-Statistic: -3.2682313237305527\n", + "P-Value: 0.04683949096861523\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.4557203389830508\n", + "T-Statistic: -7.350069795231334\n", + "P-Value: 0.00520460105096178\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 837\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 838\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.4291666666666667\n", + "T-Statistic: -3.443843878513792\n", + "P-Value: 0.041120604000996194\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3461864406779661\n", + "T-Statistic: -5.940860662234962\n", + "P-Value: 0.009534706994175631\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 838\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 839\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4435109289617486\n", + "T-Statistic: -3.1918220004780653\n", + "P-Value: 0.04964171462252754\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3879237288135593\n", + "T-Statistic: -3.2460121747166957\n", + "P-Value: 0.04763324643436874\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 839\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 840\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.45833333333333337\n", + "T-Statistic: -13.269636352208451\n", + "P-Value: 0.000924879437843544\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3924435028248588\n", + "T-Statistic: -4.124951949583921\n", + "P-Value: 0.02583406935919925\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 840\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 841\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3773224043715847\n", + "T-Statistic: -4.587084180430928\n", + "P-Value: 0.01945956097411938\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.38135593220338987\n", + "T-Statistic: -1.7503501050350099\n", + "P-Value: 0.17836241760761135\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 841\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 842\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.4666666666666667\n", + "T-Statistic: -5.570797323763848\n", + "P-Value: 0.011415660012176308\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.396680790960452\n", + "T-Statistic: -3.34946047620416\n", + "P-Value: 0.04407650716830172\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 842\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 843\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.4541666666666666\n", + "T-Statistic: -3.965300941783783\n", + "P-Value: 0.028653776872905008\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.4406779661016949\n", + "T-Statistic: -3.8340579025361636\n", + "P-Value: 0.03127462797536876\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 843\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 844\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7868852459016393\n", + "Average of Other Ratios: 0.32916666666666666\n", + "T-Statistic: -12.411848878312394\n", + "P-Value: 0.0011269497196247178\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3418079096045198\n", + "T-Statistic: -5.165514464459437\n", + "P-Value: 0.014074465381170985\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 844\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 845\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.4291666666666667\n", + "T-Statistic: -8.523579646154461\n", + "P-Value: 0.0033922878667983014\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3840395480225988\n", + "T-Statistic: -6.480569535567828\n", + "P-Value: 0.007457659131028016\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 845\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 846\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.47704918032786886\n", + "T-Statistic: -5.705858082917919\n", + "P-Value: 0.01067713705598062\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4491525423728813\n", + "T-Statistic: -6.170953464957155\n", + "P-Value: 0.008566603032878853\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 846\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 847\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.36250000000000004\n", + "T-Statistic: -4.348783893927762\n", + "P-Value: 0.022454827486827797\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.33728813559322035\n", + "T-Statistic: -2.5791808608632842\n", + "P-Value: 0.08183795390499486\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 847\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 848\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.4708333333333333\n", + "T-Statistic: -3.0287534515367915\n", + "P-Value: 0.05636661686037541\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3992467043314501\n", + "T-Statistic: -2.0879830038875222\n", + "P-Value: 0.17203960039889202\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 848\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 849\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4523224043715847\n", + "T-Statistic: -4.428901327226881\n", + "P-Value: 0.02138508188391737\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.4053672316384181\n", + "T-Statistic: -5.498322668127967\n", + "P-Value: 0.01183963469227656\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 849\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 850\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -5.497766397039839\n", + "P-Value: 0.011842967860121692\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.36292372881355933\n", + "T-Statistic: -6.299107729795289\n", + "P-Value: 0.008082951481625023\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 850\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 851\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.3694672131147541\n", + "T-Statistic: -7.27109198627894\n", + "P-Value: 0.005368618417249257\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3771186440677966\n", + "T-Statistic: -2.0909375180606187\n", + "P-Value: 0.12766278225478178\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 851\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 852\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -8.672148663324567\n", + "P-Value: 0.003226136549403669\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3423728813559322\n", + "T-Statistic: -1.7039616353344613\n", + "P-Value: 0.2305022123029592\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 852\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 853\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.3625\n", + "T-Statistic: -4.761760610530732\n", + "P-Value: 0.01758652689930633\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.32902542372881355\n", + "T-Statistic: -6.562385865517576\n", + "P-Value: 0.007196507966961928\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 853\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 854\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3647540983606557\n", + "T-Statistic: -7.531019558041822\n", + "P-Value: 0.017178611116784117\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3251412429378531\n", + "T-Statistic: -5.210719601593815\n", + "P-Value: 0.013740287056139356\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 854\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 855\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -3.1652931763476935\n", + "P-Value: 0.05066398901162397\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.32627118644067793\n", + "T-Statistic: -2.0939473213563398\n", + "P-Value: 0.17129801016782423\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 855\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 856\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.3625\n", + "T-Statistic: -6.508196721311476\n", + "P-Value: 0.007368102187221459\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.32937853107344633\n", + "T-Statistic: -3.048964780184798\n", + "P-Value: 0.05547328824097628\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 856\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 857\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4562841530054645\n", + "T-Statistic: -2.5241076101780897\n", + "P-Value: 0.08586662930511714\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.39661016949152544\n", + "T-Statistic: -7.7424126813571705\n", + "P-Value: 0.004480807680029333\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 857\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 858\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.42916666666666664\n", + "T-Statistic: -5.888028951637414\n", + "P-Value: 0.00977702506049218\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.37608286252354045\n", + "T-Statistic: -1.5525897646537226\n", + "P-Value: 0.2607163805465733\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 858\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 859\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4227459016393443\n", + "T-Statistic: -4.090863905829874\n", + "P-Value: 0.02640502342776308\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3336864406779661\n", + "T-Statistic: -3.1952478845632513\n", + "P-Value: 0.04951160636548708\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 859\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 860\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.38504098360655736\n", + "T-Statistic: -3.331454908237819\n", + "P-Value: 0.044670745752969254\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3458333333333333\n", + "T-Statistic: -9.810579653543469\n", + "P-Value: 0.002251005507538249\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 860\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 861\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4189207650273224\n", + "T-Statistic: -2.2659856668962592\n", + "P-Value: 0.10832553396730833\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.36257062146892655\n", + "T-Statistic: -4.233384268372494\n", + "P-Value: 0.0241205868939632\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 861\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 862\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4125\n", + "T-Statistic: -8.339578454332552\n", + "P-Value: 0.0036141243204565873\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.4009180790960452\n", + "T-Statistic: -4.384410769545493\n", + "P-Value: 0.02197077351423448\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 862\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 863\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -2.501174736689476\n", + "P-Value: 0.08761587052799776\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3220338983050847\n", + "T-Statistic: -5.039047529047533\n", + "P-Value: 0.015067470993325742\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 863\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 864\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.40669398907103826\n", + "T-Statistic: -7.514384861135337\n", + "P-Value: 0.004883963889775299\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.37959039548022605\n", + "T-Statistic: -3.066783460464022\n", + "P-Value: 0.05470043251723667\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 864\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 865\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.3781420765027323\n", + "T-Statistic: -5.634742497670123\n", + "P-Value: 0.011057936145282518\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.42372881355932207\n", + "T-Statistic: -2.84604989415154\n", + "P-Value: 0.06532071006198013\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 865\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 866\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3776639344262295\n", + "T-Statistic: -4.858609077032151\n", + "P-Value: 0.01664841190941072\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3458333333333333\n", + "T-Statistic: -3.64966532468076\n", + "P-Value: 0.03550063326417129\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 866\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 867\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.4625\n", + "T-Statistic: -2.1172312753822844\n", + "P-Value: 0.12450912024159524\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.358545197740113\n", + "T-Statistic: -3.7292186611760196\n", + "P-Value: 0.033592909525737194\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 867\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 868\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.43750000000000006\n", + "T-Statistic: -9.957781870946098\n", + "P-Value: 0.002154944280259222\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3461864406779661\n", + "T-Statistic: -3.667546560934897\n", + "P-Value: 0.03505999613164613\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 868\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 869\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -7.6307221538215995\n", + "P-Value: 0.004672553983451911\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.34173728813559323\n", + "T-Statistic: -3.266799462776711\n", + "P-Value: 0.046890135550055285\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 869\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 870\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4523224043715847\n", + "T-Statistic: -6.124375580236181\n", + "P-Value: 0.00875178616778322\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.40508474576271186\n", + "T-Statistic: -5.713299488454646\n", + "P-Value: 0.010638289400887548\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 870\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 871\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.34795081967213115\n", + "T-Statistic: -2.822931024840667\n", + "P-Value: 0.06657696557341944\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.2995056497175141\n", + "T-Statistic: -4.670840095116222\n", + "P-Value: 0.018530711133626115\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 871\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 872\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.3208333333333333\n", + "T-Statistic: -14.29409263370358\n", + "P-Value: 0.0007419958228994995\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3644067796610169\n", + "T-Statistic: -3.5603266600026933\n", + "P-Value: 0.03781136121703687\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 872\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 873\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -11.034070169059069\n", + "P-Value: 0.0015942956657695156\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.295409604519774\n", + "T-Statistic: -3.8525647377658387\n", + "P-Value: 0.030886920710077346\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 873\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 874\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4352459016393443\n", + "T-Statistic: -2.8995067504928964\n", + "P-Value: 0.06252731441759059\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3878531073446328\n", + "T-Statistic: -2.7019645629286955\n", + "P-Value: 0.07366341268340729\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 874\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 875\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3688524590163934\n", + "T-Statistic: -4.890622801652107\n", + "P-Value: 0.016352674036444928\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3516949152542373\n", + "T-Statistic: -2.830478177733986\n", + "P-Value: 0.06616357500554147\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 875\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 876\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.36666666666666664\n", + "T-Statistic: -4.753997634988252\n", + "P-Value: 0.01766466674069792\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.32627118644067793\n", + "T-Statistic: -2.1497076400880912\n", + "P-Value: 0.12074123791130136\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 876\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 877\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.4375\n", + "T-Statistic: -12.970212234731923\n", + "P-Value: 0.0009894935210338785\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.36285310734463283\n", + "T-Statistic: -3.674885279264579\n", + "P-Value: 0.03488117727992117\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 877\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 878\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.32916666666666666\n", + "T-Statistic: -5.592518411047395\n", + "P-Value: 0.011292471025751814\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.27415254237288134\n", + "T-Statistic: -8.239062446111248\n", + "P-Value: 0.0037434491451463214\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 878\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 879\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.39583333333333337\n", + "T-Statistic: -4.8738225751540805\n", + "P-Value: 0.016507011615195296\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.32916666666666666\n", + "T-Statistic: -28.126664663270653\n", + "P-Value: 9.866049174120721e-05\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 879\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 880\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.43333333333333335\n", + "T-Statistic: -3.1761845581166224\n", + "P-Value: 0.050241108410324004\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3347457627118644\n", + "T-Statistic: -2.188743887105837\n", + "P-Value: 0.11639121058222453\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 880\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 881\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.43749999999999994\n", + "T-Statistic: -3.491803278688525\n", + "P-Value: 0.039715098203703336\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.4218220338983051\n", + "T-Statistic: -11.810674379463547\n", + "P-Value: 0.0013048196771810819\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 881\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 882\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -3.0387597258755705\n", + "P-Value: 0.05592210337011148\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.3093220338983051\n", + "T-Statistic: -1.5714285714285712\n", + "P-Value: 0.25668888376056537\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 882\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 883\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.4458333333333333\n", + "T-Statistic: -8.515608096333422\n", + "P-Value: 0.0034015195584640207\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.47662429378531074\n", + "T-Statistic: -2.224165710326518\n", + "P-Value: 0.11260576823386355\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 883\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 884\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.45170765027322407\n", + "T-Statistic: -4.285643375079597\n", + "P-Value: 0.02334712177086421\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.396680790960452\n", + "T-Statistic: -4.238952655416746\n", + "P-Value: 0.024036610732601344\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 884\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 885\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.4147540983606558\n", + "T-Statistic: -2.7228776982622653\n", + "P-Value: 0.07237357811230395\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3206920903954802\n", + "T-Statistic: -3.753591024171325\n", + "P-Value: 0.03303482712591477\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 885\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 886\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.46509562841530055\n", + "T-Statistic: -5.141289506242194\n", + "P-Value: 0.014257918214039442\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.40946327683615824\n", + "T-Statistic: -3.475538999555497\n", + "P-Value: 0.04018478833317944\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 886\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 887\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.4208333333333334\n", + "T-Statistic: -2.4902246368970027\n", + "P-Value: 0.08846651702809234\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.379590395480226\n", + "T-Statistic: -3.6545153763584652\n", + "P-Value: 0.03538041968158101\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 887\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 888\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.43333333333333335\n", + "T-Statistic: -2.8622478469687973\n", + "P-Value: 0.06445811513039201\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.40494350282485875\n", + "T-Statistic: -2.9327458060449008\n", + "P-Value: 0.06086550016185654\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 888\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 889\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.45157103825136613\n", + "T-Statistic: -2.989953698734684\n", + "P-Value: 0.05813278125064246\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.43898305084745765\n", + "T-Statistic: -2.551058736525074\n", + "P-Value: 0.08386536725088448\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 889\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 890\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.3833333333333333\n", + "T-Statistic: -3.057286949843683\n", + "P-Value: 0.05511063014547841\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.35593220338983056\n", + "T-Statistic: -2.84815729902686\n", + "P-Value: 0.06520767196759084\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 890\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 891\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.4166666666666667\n", + "T-Statistic: -10.548278185933164\n", + "P-Value: 0.0018199105302834477\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.31645480225988704\n", + "T-Statistic: -39.57411910381416\n", + "P-Value: 3.5500916860886294e-05\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 891\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 892\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.49337431693989064\n", + "T-Statistic: -2.7692008371142\n", + "P-Value: 0.0696149257313345\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4260593220338983\n", + "T-Statistic: -8.799079181288127\n", + "P-Value: 0.0030926117525078255\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 892\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 893\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3858606557377049\n", + "T-Statistic: -3.299959459423778\n", + "P-Value: 0.0457348408632345\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3880649717514124\n", + "T-Statistic: -2.6501717516402645\n", + "P-Value: 0.07698251884827165\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 893\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 894\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.4625\n", + "T-Statistic: -3.800342933561155\n", + "P-Value: 0.031997130689119066\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.41793785310734466\n", + "T-Statistic: -3.015997915032065\n", + "P-Value: 0.05693971913913344\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 894\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 895\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3866120218579235\n", + "T-Statistic: -1.7856892936312567\n", + "P-Value: 0.21607017204333234\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.37711864406779666\n", + "T-Statistic: -3.8784935044945925\n", + "P-Value: 0.030354042786245917\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 895\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 896\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.4916666666666667\n", + "T-Statistic: -3.723879479976154\n", + "P-Value: 0.0337167759198274\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.41619585687382293\n", + "T-Statistic: -1.4090909090909103\n", + "P-Value: 0.29417736761437996\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 896\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 897\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.43948087431693994\n", + "T-Statistic: -2.3754283562972685\n", + "P-Value: 0.09802174273100749\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.39830508474576276\n", + "T-Statistic: -9.295160030897799\n", + "P-Value: 0.0026356894602544566\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 897\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 898\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.37725409836065577\n", + "T-Statistic: -3.8749319798464548\n", + "P-Value: 0.03042653470881461\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.3305084745762712\n", + "T-Statistic: -3.5988166779486694\n", + "P-Value: 0.03679295873270534\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 898\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 899\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -13.295383727881227\n", + "P-Value: 0.0009195881666894832\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3644067796610169\n", + "T-Statistic: -1.628834176702405\n", + "P-Value: 0.20183517132355724\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 899\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 900\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4316939890710383\n", + "T-Statistic: -6.669993903142867\n", + "P-Value: 0.006871037958981735\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3845338983050847\n", + "T-Statistic: -2.6313736841918445\n", + "P-Value: 0.07823278504898863\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 900\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 901\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.4916666666666667\n", + "T-Statistic: -9.672172825695142\n", + "P-Value: 0.0023465711838438533\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.4138418079096045\n", + "T-Statistic: -3.773628023053654\n", + "P-Value: 0.032584885614146145\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 901\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 902\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.4666666666666667\n", + "T-Statistic: -9.041003671038775\n", + "P-Value: 0.0028577010889647217\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3712806026365348\n", + "T-Statistic: -1.8625711948272987\n", + "P-Value: 0.20356174355735687\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 902\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 903\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4603825136612022\n", + "T-Statistic: -3.7003174802745145\n", + "P-Value: 0.03427044013621927\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.409180790960452\n", + "T-Statistic: -4.9902021373122745\n", + "P-Value: 0.015475409518374506\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 903\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 904\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7666666666666667\n", + "Average of Other Ratios: 0.3653005464480874\n", + "T-Statistic: -8.367710625283678\n", + "P-Value: 0.0035789897552198605\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3536723163841808\n", + "T-Statistic: -3.085580966165473\n", + "P-Value: 0.09093394434924655\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 904\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 905\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -11.604959601054722\n", + "P-Value: 0.0013742030324602752\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3163841807909604\n", + "T-Statistic: -10.370899457402706\n", + "P-Value: 0.0019128148288002538\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 905\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 906\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.4771174863387978\n", + "T-Statistic: -4.574500631486023\n", + "P-Value: 0.019604293549578128\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.40070621468926554\n", + "T-Statistic: -5.159875560151832\n", + "P-Value: 0.014116891909403921\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 906\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 907\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.39999999999999997\n", + "T-Statistic: -4.521534453338067\n", + "P-Value: 0.020229034371625897\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.33319209039548026\n", + "T-Statistic: -4.493884775113415\n", + "P-Value: 0.02056543411778116\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 907\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 908\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4359972677595628\n", + "T-Statistic: -3.71598604315986\n", + "P-Value: 0.03390097325869757\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4152542372881356\n", + "T-Statistic: -2.847055192728138\n", + "P-Value: 0.0652667568561372\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 908\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 909\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -4.12972900744298\n", + "P-Value: 0.025755332738221934\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.33145009416195853\n", + "T-Statistic: -11.716898663286049\n", + "P-Value: 0.007205446408459767\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 909\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 910\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.35416666666666663\n", + "T-Statistic: -11.65895831862298\n", + "P-Value: 0.0013555253490377533\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.29936440677966103\n", + "T-Statistic: -1.9119123291898108\n", + "P-Value: 0.1518400483442069\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 910\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 911\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -8.407323638843176\n", + "P-Value: 0.0035302782268764117\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.32005649717514123\n", + "T-Statistic: -2.143768150852684\n", + "P-Value: 0.1652700646335296\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 911\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 912\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.32499999999999996\n", + "T-Statistic: -8.404697750055918\n", + "P-Value: 0.003533480017692563\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.2693973634651601\n", + "T-Statistic: -2.180850112451567\n", + "P-Value: 0.1609698801291486\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 912\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 913\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.47704918032786886\n", + "T-Statistic: -3.572872262389389\n", + "P-Value: 0.037475510464252\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4132768361581921\n", + "T-Statistic: -2.677380542667445\n", + "P-Value: 0.07521629738306508\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 913\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 914\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -3.3807925592909616\n", + "P-Value: 0.04306616942393178\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3771186440677966\n", + "T-Statistic: -7.008990210800726\n", + "P-Value: 0.005964312251936928\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 914\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 915\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.43558743169398906\n", + "T-Statistic: -4.55116556463313\n", + "P-Value: 0.01987640371413144\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.576271186440678\n", + "Average of Other Ratios: 0.400635593220339\n", + "T-Statistic: -6.348003491637961\n", + "P-Value: 0.007907886129610865\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 915\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 916\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.5\n", + "T-Statistic: -3.265341686400343\n", + "P-Value: 0.04694176800449449\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3949152542372881\n", + "T-Statistic: -18.999999999999993\n", + "P-Value: 0.03347541671314822\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 916\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 917\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.42711748633879776\n", + "T-Statistic: -3.8576108132230886\n", + "P-Value: 0.030782280046241853\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3474576271186441\n", + "T-Statistic: -3.086709862908689\n", + "P-Value: 0.05385213329290739\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 917\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 918\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4313524590163934\n", + "T-Statistic: -3.8365066246858697\n", + "P-Value: 0.03122297154382187\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3258003766478343\n", + "T-Statistic: -15.805266578356209\n", + "P-Value: 0.003979221225814003\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 918\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 919\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.3458333333333333\n", + "T-Statistic: -11.442187082108672\n", + "P-Value: 0.0014326132345143508\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3459039548022599\n", + "T-Statistic: -2.838910304269998\n", + "P-Value: 0.06570548867709089\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 919\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 920\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4420765027322404\n", + "T-Statistic: -4.718756309978549\n", + "P-Value: 0.04209478615042156\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4134887005649718\n", + "T-Statistic: -3.585413143844187\n", + "P-Value: 0.037143581857500066\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 920\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 921\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.45266393442622954\n", + "T-Statistic: -5.549004690060408\n", + "P-Value: 0.011541028000440172\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.43848870056497175\n", + "T-Statistic: -3.2975199061269667\n", + "P-Value: 0.04581859729168717\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 921\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 922\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.4666666666666667\n", + "T-Statistic: -4.2869837090014355\n", + "P-Value: 0.02332770866411145\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4009180790960452\n", + "T-Statistic: -4.901036565932749\n", + "P-Value: 0.016257948615872354\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 922\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 923\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.4291666666666667\n", + "T-Statistic: -3.7406497535262835\n", + "P-Value: 0.03332966691975382\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3584039548022599\n", + "T-Statistic: -3.1383427887418938\n", + "P-Value: 0.051729870414491945\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 923\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 924\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7333333333333333\n", + "Average of Other Ratios: 0.5189890710382514\n", + "T-Statistic: -7.795621704107285\n", + "P-Value: 0.004393108665011521\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.5021892655367232\n", + "T-Statistic: -6.111799193237621\n", + "P-Value: 0.008802686470096136\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 924\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 925\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3902322404371585\n", + "T-Statistic: -7.19301070709552\n", + "P-Value: 0.005537540195779914\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -6.884772011368271\n", + "P-Value: 0.00627716185131719\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 925\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 926\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.360724043715847\n", + "T-Statistic: -7.579568529572813\n", + "P-Value: 0.004763992639250445\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3459039548022599\n", + "T-Statistic: -2.5149332127933626\n", + "P-Value: 0.08656123080121365\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 926\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 927\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7333333333333333\n", + "Average of Other Ratios: 0.3858606557377049\n", + "T-Statistic: -14.089106567555575\n", + "P-Value: 0.0007744612617815288\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.39668079096045206\n", + "T-Statistic: -3.806413998599051\n", + "P-Value: 0.03186546090943339\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 927\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 928\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -2.527395473158105\n", + "P-Value: 0.08561936927249675\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.34187853107344635\n", + "T-Statistic: -4.207255276582219\n", + "P-Value: 0.02451973533970688\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 928\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 929\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.3875\n", + "T-Statistic: -12.007025761124119\n", + "P-Value: 0.001242867214816079\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3376412429378531\n", + "T-Statistic: -4.4091197996331335\n", + "P-Value: 0.021642966336736617\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 929\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 930\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4230874316939891\n", + "T-Statistic: -3.347260874883563\n", + "P-Value: 0.04414856066329664\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3728813559322034\n", + "T-Statistic: -2.2290356659326926\n", + "P-Value: 0.11209698013636105\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 930\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 931\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7540983606557377\n", + "Average of Other Ratios: 0.35000000000000003\n", + "T-Statistic: -16.471846559014978\n", + "P-Value: 0.00048698021922821526\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3474576271186441\n", + "T-Statistic: -3.0867098629086884\n", + "P-Value: 0.05385213329290739\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 931\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 932\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.38749999999999996\n", + "T-Statistic: -39.783878716039034\n", + "P-Value: 3.4943182204626624e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3728813559322034\n", + "T-Statistic: -3.7150594432211537\n", + "P-Value: 0.033922680348442695\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 932\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 933\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.3833333333333333\n", + "T-Statistic: -27.287366605133304\n", + "P-Value: 0.0001080165038157327\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3629943502824859\n", + "T-Statistic: -3.5263157894736845\n", + "P-Value: 0.03874136219588886\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 933\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 934\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3605874316939891\n", + "T-Statistic: -4.472129156189748\n", + "P-Value: 0.020835236225830767\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3627824858757063\n", + "T-Statistic: -5.7192755125875685\n", + "P-Value: 0.010607225660250866\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 934\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 935\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.42083333333333334\n", + "T-Statistic: -6.50235424552611\n", + "P-Value: 0.007386922898311131\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.2923728813559322\n", + "T-Statistic: -10.891044814105634\n", + "P-Value: 0.001656669068269899\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 935\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 936\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.36502732240437163\n", + "T-Statistic: -7.572061952950856\n", + "P-Value: 0.004777609010065385\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.34173728813559323\n", + "T-Statistic: -17.461538461538474\n", + "P-Value: 0.0004093729804445589\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 936\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 937\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.425\n", + "T-Statistic: -4.356743170503809\n", + "P-Value: 0.022345498848743366\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4279661016949152\n", + "T-Statistic: -4.429570467592002\n", + "P-Value: 0.021376428303786093\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 937\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 938\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4153688524590164\n", + "T-Statistic: -3.1703304743958\n", + "P-Value: 0.05046784910179293\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.33319209039548026\n", + "T-Statistic: -1.7611959878594554\n", + "P-Value: 0.17642304434957817\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 938\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 939\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3284153005464481\n", + "T-Statistic: -3.071040072795025\n", + "P-Value: 0.05451781694215867\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.2966101694915254\n", + "T-Statistic: -4.147575310031268\n", + "P-Value: 0.025463909186015793\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 939\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 940\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.4375\n", + "T-Statistic: -5.097326714872012\n", + "P-Value: 0.014598887274167346\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.4387005649717514\n", + "T-Statistic: -6.879715746378599\n", + "P-Value: 0.006290349260396344\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 940\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 941\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.39781420765027325\n", + "T-Statistic: -16.249999999999922\n", + "P-Value: 0.0037656052779771634\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3545197740112994\n", + "T-Statistic: -10.777777777777775\n", + "P-Value: 0.0017083811519259897\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 941\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 942\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.35947176684881604\n", + "T-Statistic: -3.9829601606847738\n", + "P-Value: 0.05763976433444822\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.32895480225988705\n", + "T-Statistic: -4.070774182545795\n", + "P-Value: 0.026749143973066495\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 942\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 943\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.4313524590163934\n", + "T-Statistic: -1.9917554271299593\n", + "P-Value: 0.14044440711390466\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.40254237288135597\n", + "T-Statistic: -2.414985892866683\n", + "P-Value: 0.09459301523161902\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 943\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 944\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.41905737704918034\n", + "T-Statistic: -7.182256405781872\n", + "P-Value: 0.005561352568453646\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.4221751412429378\n", + "T-Statistic: -3.7480916030534375\n", + "P-Value: 0.03315971002355427\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 944\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 945\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3982923497267759\n", + "T-Statistic: -5.599507245339936\n", + "P-Value: 0.011253204715190618\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.37139830508474575\n", + "T-Statistic: -6.762389833500976\n", + "P-Value: 0.006606867955287357\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 945\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 946\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.39829234972677596\n", + "T-Statistic: -4.723689577243244\n", + "P-Value: 0.017974094449026225\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.33742937853107347\n", + "T-Statistic: -4.058782494615296\n", + "P-Value: 0.02695730866033387\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 946\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 947\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.38333333333333336\n", + "T-Statistic: -4.2565386541927595\n", + "P-Value: 0.02377386602678355\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3629943502824859\n", + "T-Statistic: -2.8181818181818175\n", + "P-Value: 0.06683875138704513\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 947\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 948\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.42499999999999993\n", + "T-Statistic: -3.2030019695384855\n", + "P-Value: 0.04921870916506516\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4152542372881356\n", + "T-Statistic: -2.558118456493597\n", + "P-Value: 0.08335065260117518\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 948\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 949\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.3875\n", + "T-Statistic: -1.7671941136183904\n", + "P-Value: 0.1753608911595616\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.2911016949152542\n", + "T-Statistic: -2.67070108252106\n", + "P-Value: 0.07564520129729006\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 949\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 950\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.42500000000000004\n", + "T-Statistic: -5.444705021331377\n", + "P-Value: 0.01216663456000868\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.37139830508474575\n", + "T-Statistic: -5.336986813075771\n", + "P-Value: 0.012860119949256482\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 950\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 951\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.3375\n", + "T-Statistic: -4.229541871678541\n", + "P-Value: 0.024178754668284275\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.2701271186440678\n", + "T-Statistic: -4.975196209154729\n", + "P-Value: 0.015603617644457647\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 951\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 952\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.48087431693989074\n", + "T-Statistic: -2.870168301345979\n", + "P-Value: 0.06404150157384371\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4682909604519774\n", + "T-Statistic: -4.883695241304617\n", + "P-Value: 0.01641608650487274\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 952\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 953\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.3958333333333333\n", + "T-Statistic: -8.130635229762294\n", + "P-Value: 0.003889900792911817\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.34985875706214686\n", + "T-Statistic: -2.370422697154792\n", + "P-Value: 0.09846632715247584\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 953\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 954\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.4541666666666667\n", + "T-Statistic: -4.048412402739574\n", + "P-Value: 0.02713900943382794\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.39272598870056497\n", + "T-Statistic: -3.3006539570698283\n", + "P-Value: 0.04571103238074162\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 954\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 955\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4519808743169399\n", + "T-Statistic: -2.7715039029613733\n", + "P-Value: 0.0694812123937475\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4322033898305085\n", + "T-Statistic: -4.815101245041479\n", + "P-Value: 0.017061585926028165\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 955\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 956\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.4562841530054645\n", + "T-Statistic: -6.639086391609726\n", + "P-Value: 0.006962502866864333\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.4262005649717514\n", + "T-Statistic: -4.051889951012785\n", + "P-Value: 0.027077901844717624\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 956\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 957\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3402322404371585\n", + "T-Statistic: -5.379181269835038\n", + "P-Value: 0.012582469390043725\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.35169491525423724\n", + "T-Statistic: -2.9151315524126016\n", + "P-Value: 0.06173916082876997\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 957\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 958\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4125\n", + "T-Statistic: -3.2184309034451113\n", + "P-Value: 0.04864239224720965\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3755649717514124\n", + "T-Statistic: -10.56303549746513\n", + "P-Value: 0.0018124532293534081\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 958\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 959\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3690573770491803\n", + "T-Statistic: -6.0781372785179055\n", + "P-Value: 0.008940851611301535\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.2911016949152542\n", + "T-Statistic: -10.742132190804174\n", + "P-Value: 0.0017250974273507926\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 959\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 960\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -11.123418935025548\n", + "P-Value: 0.0015569047785033382\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3686440677966102\n", + "T-Statistic: -2.812229917701684\n", + "P-Value: 0.06716864584643882\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 960\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 961\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.38749999999999996\n", + "T-Statistic: -8.52694846917119\n", + "P-Value: 0.0033883964608289563\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.37570621468926557\n", + "T-Statistic: -5.0089472186085136\n", + "P-Value: 0.015317175750995964\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 961\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 962\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4227459016393443\n", + "T-Statistic: -3.8587298371444883\n", + "P-Value: 0.030759136480267034\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3753531073446328\n", + "T-Statistic: -3.3382526588579973\n", + "P-Value: 0.044445211846365185\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 962\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 963\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4936475409836066\n", + "T-Statistic: -2.5087837781411735\n", + "P-Value: 0.08703067245521756\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4302966101694915\n", + "T-Statistic: -2.890628912501168\n", + "P-Value: 0.06298074227793468\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 963\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 964\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.39583333333333337\n", + "T-Statistic: -21.672082554883993\n", + "P-Value: 0.00021500571596787181\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.39830508474576276\n", + "T-Statistic: -3.185621103989269\n", + "P-Value: 0.049878312712040275\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 964\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 965\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4441256830601093\n", + "T-Statistic: -4.114953099885994\n", + "P-Value: 0.025999880249744926\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.35868644067796607\n", + "T-Statistic: -5.327850954897623\n", + "P-Value: 0.012921287882286202\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 965\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 966\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.3152322404371585\n", + "T-Statistic: -27.094781124739157\n", + "P-Value: 0.00011032862635579921\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.31207627118644066\n", + "T-Statistic: -3.533152576889349\n", + "P-Value: 0.03855209262090739\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 966\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 967\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.819672131147541\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -9.79685729377205\n", + "P-Value: 0.002260247404737733\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3608757062146893\n", + "T-Statistic: -1.4303797468354427\n", + "P-Value: 0.38842195960522763\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 967\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 968\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.325\n", + "T-Statistic: -4.952772918130208\n", + "P-Value: 0.015797788968726655\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.29555084745762716\n", + "T-Statistic: -7.084467611285733\n", + "P-Value: 0.005784199991243675\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 968\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 969\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -7.3460047131317\n", + "P-Value: 0.00521288063379473\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3987758945386064\n", + "T-Statistic: -5.717277635038901\n", + "P-Value: 0.029257011705986744\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 969\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 970\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3919854280510018\n", + "T-Statistic: -1.2592474439830368\n", + "P-Value: 0.33499662016654086\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.32888418079096043\n", + "T-Statistic: -2.287766368648155\n", + "P-Value: 0.10617410540553696\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 970\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 971\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.41250000000000003\n", + "T-Statistic: -6.491699980107792\n", + "P-Value: 0.00742140717977802\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.42627118644067796\n", + "T-Statistic: -3.519233773599045\n", + "P-Value: 0.03893867159865643\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 971\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 972\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.39583333333333337\n", + "T-Statistic: -16.47597884542129\n", + "P-Value: 0.0004866171022863723\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3332627118644068\n", + "T-Statistic: -7.238793509124532\n", + "P-Value: 0.005437657782298041\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 972\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 973\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.4605874316939891\n", + "T-Statistic: -9.319488874116459\n", + "P-Value: 0.002615651895897316\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4092514124293785\n", + "T-Statistic: -6.2886655641721925\n", + "P-Value: 0.008120996607005396\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 973\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 974\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.37725409836065577\n", + "T-Statistic: -9.746825292040107\n", + "P-Value: 0.0022943725355199732\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.4025423728813559\n", + "T-Statistic: -2.8010960513215215\n", + "P-Value: 0.06779122416384425\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 974\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 975\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4519125683060109\n", + "T-Statistic: -4.467667765204196\n", + "P-Value: 0.020891129917487965\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.40473163841807913\n", + "T-Statistic: -3.46978940590354\n", + "P-Value: 0.040352519825929224\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 975\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 976\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.3833333333333333\n", + "T-Statistic: -3.8203179931250597\n", + "P-Value: 0.03156651788894358\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3502824858757062\n", + "T-Statistic: -4.196397990844169\n", + "P-Value: 0.024688099628686736\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 976\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 977\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7540983606557377\n", + "Average of Other Ratios: 0.41250000000000003\n", + "T-Statistic: -6.306431273644387\n", + "P-Value: 0.008056408388901619\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.417725988700565\n", + "T-Statistic: -7.10832364958241\n", + "P-Value: 0.005728765340993184\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 977\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 978\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.4853142076502732\n", + "T-Statistic: -5.664156791225322\n", + "P-Value: 0.010898310561203092\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.46836158192090394\n", + "T-Statistic: -9.941988596598865\n", + "P-Value: 0.0021649872660866308\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 978\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 979\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.35239071038251363\n", + "T-Statistic: -10.603077839669773\n", + "P-Value: 0.001792421776010322\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.2995762711864407\n", + "T-Statistic: -10.04928647585347\n", + "P-Value: 0.00209795503470734\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 979\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 980\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.42288251366120216\n", + "T-Statistic: -2.792068808043707\n", + "P-Value: 0.06830128777723389\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3220338983050848\n", + "T-Statistic: -3.6987295116025987\n", + "P-Value: 0.03430817170101989\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 980\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 981\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -6.797930999977556\n", + "P-Value: 0.00650881472279479\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.33312146892655364\n", + "T-Statistic: -2.347903032601037\n", + "P-Value: 0.1004970270572768\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 981\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 982\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.4416666666666667\n", + "T-Statistic: -2.8952739132187233\n", + "P-Value: 0.06274299207471501\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.3836864406779661\n", + "T-Statistic: -3.485462485693465\n", + "P-Value: 0.03989737572645941\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 982\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 983\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4230191256830601\n", + "T-Statistic: -3.5968393291702223\n", + "P-Value: 0.03684441767090311\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3925141242937853\n", + "T-Statistic: -3.5865984774856337\n", + "P-Value: 0.03711240322472348\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 983\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 984\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.36495901639344264\n", + "T-Statistic: -8.629044905647506\n", + "P-Value: 0.003273211490827028\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.326271186440678\n", + "T-Statistic: -3.788162541206022\n", + "P-Value: 0.03226341112647686\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 984\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 985\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -2.9757050833898977\n", + "P-Value: 0.05879878128106479\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.3923728813559322\n", + "T-Statistic: -20.347964212104664\n", + "P-Value: 0.00025950431091567957\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 985\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 986\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.4666666666666667\n", + "T-Statistic: -15.055959655317555\n", + "P-Value: 0.0006360495232655051\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.4134887005649717\n", + "T-Statistic: -3.901282639398381\n", + "P-Value: 0.029895400966674732\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 986\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 987\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.42083333333333334\n", + "T-Statistic: -3.1359127797177497\n", + "P-Value: 0.05182736123156866\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3983050847457627\n", + "T-Statistic: -5.867173552077365\n", + "P-Value: 0.009874902457062155\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 987\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 988\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.36666666666666664\n", + "T-Statistic: -12.510329824209249\n", + "P-Value: 0.0011009408982364437\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.3220338983050848\n", + "T-Statistic: -2.4570411640937726\n", + "P-Value: 0.09110685313385442\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 988\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 989\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.4400273224043716\n", + "T-Statistic: -4.731321598151042\n", + "P-Value: 0.017895516622468247\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.40254237288135597\n", + "T-Statistic: -3.2790242451070704\n", + "P-Value: 0.04645996816866966\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 989\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 990\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.41666666666666663\n", + "T-Statistic: -8.004718441897252\n", + "P-Value: 0.004069631315123374\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.38418079096045193\n", + "T-Statistic: -4.595745763298827\n", + "P-Value: 0.019360742557531358\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 990\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 991\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -10.34387657107953\n", + "P-Value: 0.0019275165468191863\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3940677966101695\n", + "T-Statistic: -4.750402896977879\n", + "P-Value: 0.017701003201901513\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 991\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 992\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4271174863387978\n", + "T-Statistic: -4.102891976310035\n", + "P-Value: 0.026201720340923294\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.363135593220339\n", + "T-Statistic: -4.5211227964641365\n", + "P-Value: 0.02023399027426734\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 992\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 993\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4565573770491803\n", + "T-Statistic: -3.871284033511624\n", + "P-Value: 0.030501016317830993\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.4175141242937853\n", + "T-Statistic: -2.8647998103065557\n", + "P-Value: 0.06432351407478958\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 993\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 994\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -7.362230013436795\n", + "P-Value: 0.005179936955568627\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3798022598870056\n", + "T-Statistic: -14.122758056462086\n", + "P-Value: 0.0007690037706351588\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 994\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 995\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -5.629951970401516\n", + "P-Value: 0.01108422263908312\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.33764124293785314\n", + "T-Statistic: -3.4045365361503497\n", + "P-Value: 0.042320045665658175\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 995\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 996\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.38989071038251366\n", + "T-Statistic: -7.027749672525287\n", + "P-Value: 0.005918864007946058\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3712570621468927\n", + "T-Statistic: -6.263567998756802\n", + "P-Value: 0.008213404490252622\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 996\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 997\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4684426229508197\n", + "T-Statistic: -3.0715294425556663\n", + "P-Value: 0.05449687138415423\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4639830508474576\n", + "T-Statistic: -3.2766095606229877\n", + "P-Value: 0.04654453906356892\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 997\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 998\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3942622950819672\n", + "T-Statistic: -5.155433917254585\n", + "P-Value: 0.01415042791070789\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3123587570621469\n", + "T-Statistic: -5.08813713983205\n", + "P-Value: 0.01467150017877107\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 998\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 999\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4093806921675774\n", + "T-Statistic: -2.1811187398383804\n", + "P-Value: 0.16093929016287928\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3728813559322034\n", + "T-Statistic: -3.5693253330753327\n", + "P-Value: 0.03757007584643612\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 999\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1000\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4518442622950819\n", + "T-Statistic: -2.7116536407871275\n", + "P-Value: 0.07306231680161718\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.364406779661017\n", + "T-Statistic: -5.27455297545764\n", + "P-Value: 0.01328581900289993\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1000\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1001\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.38995901639344266\n", + "T-Statistic: -4.717870559277373\n", + "P-Value: 0.01803430740365783\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.35451977401129947\n", + "T-Statistic: -6.7777777777777715\n", + "P-Value: 0.006564177674076621\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1001\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1002\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.33572404371584696\n", + "T-Statistic: -5.92922902548398\n", + "P-Value: 0.009587374714266027\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.27118644067796605\n", + "T-Statistic: -2.510197384488104\n", + "P-Value: 0.0869224831963583\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1002\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1003\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7540983606557377\n", + "Average of Other Ratios: 0.31666666666666665\n", + "T-Statistic: -8.09966109145276\n", + "P-Value: 0.0039331240116504485\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.30338983050847457\n", + "T-Statistic: -5.263055373168918\n", + "P-Value: 0.034257089055649254\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1003\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1004\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3855191256830601\n", + "T-Statistic: -2.7763365971020724\n", + "P-Value: 0.06920166849483687\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3077683615819209\n", + "T-Statistic: -1.8092905922289133\n", + "P-Value: 0.16810978065452678\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1004\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1005\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -13.376800824017138\n", + "P-Value: 0.0009031187936782386\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.33742937853107347\n", + "T-Statistic: -2.129023252677582\n", + "P-Value: 0.12312500629142373\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1005\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1006\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3734972677595629\n", + "T-Statistic: -2.323557226050761\n", + "P-Value: 0.10274990807505914\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.3163841807909604\n", + "T-Statistic: -4.041451884327386\n", + "P-Value: 0.02726185440936123\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1006\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1007\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4319672131147541\n", + "T-Statistic: -3.959212335659781\n", + "P-Value: 0.028768976615567175\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4301553672316384\n", + "T-Statistic: -2.476997933163102\n", + "P-Value: 0.08950755799392057\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1007\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1008\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3937841530054645\n", + "T-Statistic: -3.3540166875307085\n", + "P-Value: 0.04392773109730455\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3305084745762712\n", + "T-Statistic: -3.9024896268095732\n", + "P-Value: 0.029871358819325514\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1008\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1009\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.3860655737704918\n", + "T-Statistic: -6.951550947054901\n", + "P-Value: 0.006106360263638599\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.36687853107344637\n", + "T-Statistic: -2.115310434853906\n", + "P-Value: 0.12473633813156884\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1009\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1010\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.3875\n", + "T-Statistic: -11.40102880068347\n", + "P-Value: 0.0014479028367797866\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.40487288135593225\n", + "T-Statistic: -4.138026787575419\n", + "P-Value: 0.025619300883706696\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1010\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1011\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.43572404371584705\n", + "T-Statistic: -11.348123445291295\n", + "P-Value: 0.0014678760475146785\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3541431261770245\n", + "T-Statistic: -2.568187024807373\n", + "P-Value: 0.1240305945387\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1011\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1012\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.2875\n", + "T-Statistic: -27.500378882331333\n", + "P-Value: 0.00010553371875927762\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.29103107344632767\n", + "T-Statistic: -3.7674177162295606\n", + "P-Value: 0.03272349692653532\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1012\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1013\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.30683060109289617\n", + "T-Statistic: -10.923824320280426\n", + "P-Value: 0.0016420917950501913\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.26151129943502827\n", + "T-Statistic: -3.8711758641214336\n", + "P-Value: 0.03050322842853845\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1013\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1014\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.3125\n", + "T-Statistic: -3.893753295658278\n", + "P-Value: 0.03004594161105454\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.2489406779661017\n", + "T-Statistic: -6.249324287797364\n", + "P-Value: 0.008266464021748235\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1014\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1015\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.3523907103825137\n", + "T-Statistic: -6.8536427559842705\n", + "P-Value: 0.006358933335598355\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3287429378531073\n", + "T-Statistic: -1.9757525677542294\n", + "P-Value: 0.1426456503116112\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1015\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1016\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4937158469945355\n", + "T-Statistic: -4.691276871564279\n", + "P-Value: 0.018312849057309264\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.46002824858757063\n", + "T-Statistic: -3.229172176597325\n", + "P-Value: 0.048246204514222006\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1016\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1017\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.33333333333333337\n", + "T-Statistic: -5.788824671837737\n", + "P-Value: 0.010254290938715958\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.27019774011299436\n", + "T-Statistic: -5.5858638786774515\n", + "P-Value: 0.011330025903708709\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1017\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1018\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3273224043715847\n", + "T-Statistic: -5.436414841751896\n", + "P-Value: 0.012218244055546734\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.26596045197740115\n", + "T-Statistic: -3.6973851037027248\n", + "P-Value: 0.03434015761009596\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1018\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1019\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.3333333333333333\n", + "T-Statistic: -3.361655189247521\n", + "P-Value: 0.043679734758324804\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.274364406779661\n", + "T-Statistic: -4.1604734938620584\n", + "P-Value: 0.02525593036334952\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1019\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1020\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -2.9532447616676007\n", + "P-Value: 0.059868156846439195\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.3347457627118644\n", + "T-Statistic: -1.9898190482959088\n", + "P-Value: 0.14070862393531006\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1020\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1021\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.43333333333333335\n", + "T-Statistic: -4.127992550132961\n", + "P-Value: 0.025783917599408773\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.35437853107344636\n", + "T-Statistic: -2.9292618143712588\n", + "P-Value: 0.06103707112726652\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1021\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1022\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3692622950819672\n", + "T-Statistic: -8.104386708891786\n", + "P-Value: 0.003926488707999252\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3251412429378531\n", + "T-Statistic: -2.3178660050430633\n", + "P-Value: 0.10328535861126524\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1022\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1023\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -4.309973795297764\n", + "P-Value: 0.022997950137582556\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3207627118644068\n", + "T-Statistic: -9.7408025920138\n", + "P-Value: 0.002298526341652219\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1023\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1024\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3651639344262295\n", + "T-Statistic: -3.2122265752036507\n", + "P-Value: 0.04887311160481489\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.2786723163841808\n", + "T-Statistic: -7.110897442499789\n", + "P-Value: 0.005722826489597031\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1024\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1025\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.39002732240437155\n", + "T-Statistic: -3.9886831470208737\n", + "P-Value: 0.02821684888437735\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.32514124293785307\n", + "T-Statistic: -3.3894381008331114\n", + "P-Value: 0.04279257405370464\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1025\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1026\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.3458333333333333\n", + "T-Statistic: -4.2504183551249355\n", + "P-Value: 0.02386488383717604\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.31631355932203387\n", + "T-Statistic: -4.476394699131555\n", + "P-Value: 0.020781977238118643\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1026\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1027\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.32499999999999996\n", + "T-Statistic: -6.880879437718984\n", + "P-Value: 0.006287310969453765\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.27838983050847455\n", + "T-Statistic: -1.909090909090913\n", + "P-Value: 0.15226214753320294\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1027\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1028\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4148907103825137\n", + "T-Statistic: -6.273699658520553\n", + "P-Value: 0.008175934849579894\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.32090395480225986\n", + "T-Statistic: -4.966780972281921\n", + "P-Value: 0.015676121865843336\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1028\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1029\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -2.754702614168314\n", + "P-Value: 0.07046406236659389\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.36278248587570616\n", + "T-Statistic: -8.000108500500483\n", + "P-Value: 0.004076417682476059\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1029\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1030\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.3958333333333333\n", + "T-Statistic: -6.7433095204558455\n", + "P-Value: 0.006660310850275737\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3771186440677966\n", + "T-Statistic: -3.8784935044945983\n", + "P-Value: 0.030354042786245806\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1030\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1031\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.38749999999999996\n", + "T-Statistic: -4.020027050959187\n", + "P-Value: 0.02764449995869553\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3313559322033898\n", + "T-Statistic: -4.454545454545456\n", + "P-Value: 0.04688018499081239\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1031\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1032\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4400273224043716\n", + "T-Statistic: -2.9245926757956453\n", + "P-Value: 0.061267955766122074\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3502824858757062\n", + "T-Statistic: -3.8334908600273225\n", + "P-Value: 0.03128660551954041\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1032\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1033\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -7.524590163934425\n", + "P-Value: 0.004864919741574715\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.3545197740112994\n", + "T-Statistic: -6.51303802142941\n", + "P-Value: 0.0073525543697032375\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1033\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1034\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7333333333333333\n", + "Average of Other Ratios: 0.28647540983606556\n", + "T-Statistic: -23.29092393225503\n", + "P-Value: 0.00017339475386319076\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.2870762711864407\n", + "T-Statistic: -4.234130868902486\n", + "P-Value: 0.024109305527904665\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1034\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1035\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.3458333333333333\n", + "T-Statistic: -3.5424739195231005\n", + "P-Value: 0.038295935976556825\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.29950564971751414\n", + "T-Statistic: -1.737938290656899\n", + "P-Value: 0.18061180344428132\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1035\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1036\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.36099726775956287\n", + "T-Statistic: -6.160098807321899\n", + "P-Value: 0.008609295466603118\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.288135593220339\n", + "T-Statistic: -2.647722058442962\n", + "P-Value: 0.07714404224936885\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1036\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1037\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.4400956284153006\n", + "T-Statistic: -2.282897070641292\n", + "P-Value: 0.10665056460060143\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.35466101694915253\n", + "T-Statistic: -7.02229226282608\n", + "P-Value: 0.005932038385975604\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1037\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1038\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.41994535519125686\n", + "T-Statistic: -18.636835301251896\n", + "P-Value: 0.0028667184146571527\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3632062146892655\n", + "T-Statistic: -2.200120786201923\n", + "P-Value: 0.1151589518445011\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1038\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1039\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4271174863387978\n", + "T-Statistic: -5.673540790894663\n", + "P-Value: 0.010848020755982598\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.35444915254237286\n", + "T-Statistic: -4.143365599990044\n", + "P-Value: 0.025532267312653276\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1039\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1040\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.44405737704918036\n", + "T-Statistic: -3.2047054428325206\n", + "P-Value: 0.04915465698343356\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3882062146892655\n", + "T-Statistic: -2.937104220060188\n", + "P-Value: 0.060651717076612384\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1040\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1041\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.3625\n", + "T-Statistic: -11.657726867446977\n", + "P-Value: 0.0013559475300634065\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3415960451977401\n", + "T-Statistic: -3.9152539744439387\n", + "P-Value: 0.029618618166276877\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1041\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1042\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.36919398907103823\n", + "T-Statistic: -6.079597077972364\n", + "P-Value: 0.008934801065512471\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3203389830508475\n", + "T-Statistic: -4.2850226450669\n", + "P-Value: 0.023356119365012706\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1042\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1043\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.3416666666666666\n", + "T-Statistic: -39.859982700803556\n", + "P-Value: 3.47437157745963e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.34173728813559323\n", + "T-Statistic: -5.289174189736959\n", + "P-Value: 0.013184494475533716\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1043\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1044\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.33749999999999997\n", + "T-Statistic: -9.2248243559719\n", + "P-Value: 0.0026947741822498272\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.29103107344632767\n", + "T-Statistic: -3.585517945697556\n", + "P-Value: 0.03714082383617849\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1044\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1045\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.4937158469945355\n", + "T-Statistic: -8.461896046771958\n", + "P-Value: 0.0034645967635944996\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.48128531073446335\n", + "T-Statistic: -3.3967856291561485\n", + "P-Value: 0.04256178912971218\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1045\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1046\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.33551912568306014\n", + "T-Statistic: -6.233747163298384\n", + "P-Value: 0.008325007314823952\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3377824858757062\n", + "T-Statistic: -3.3339462897065575\n", + "P-Value: 0.04458791920005854\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1046\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1047\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.36249999999999993\n", + "T-Statistic: -12.541115855481232\n", + "P-Value: 0.0010929740797637178\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.34745762711864403\n", + "T-Statistic: -3.119513763428299\n", + "P-Value: 0.052491375678345144\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1047\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1048\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.39166666666666666\n", + "T-Statistic: -3.4744562728924717\n", + "P-Value: 0.040216306514409425\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3121468926553672\n", + "T-Statistic: -9.141660034508423\n", + "P-Value: 0.00276691986404339\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1048\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1049\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.40204918032786885\n", + "T-Statistic: -2.291468930292345\n", + "P-Value: 0.10581353287285375\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.3305084745762712\n", + "T-Statistic: -10.796450033846\n", + "P-Value: 0.0016997103571064131\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1049\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1050\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.42500000000000004\n", + "T-Statistic: -3.9774734463808428\n", + "P-Value: 0.028425239599828966\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3672316384180791\n", + "T-Statistic: -2.274140974459845\n", + "P-Value: 0.10751386463711397\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1050\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1051\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.41898907103825134\n", + "T-Statistic: -7.392993166333208\n", + "P-Value: 0.005118225808922169\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4007768361581921\n", + "T-Statistic: -6.242283371885267\n", + "P-Value: 0.008292858606378618\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1051\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1052\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.42732240437158475\n", + "T-Statistic: -2.616902813780725\n", + "P-Value: 0.07921236839961415\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.37711864406779666\n", + "T-Statistic: -8.982407047314\n", + "P-Value: 0.0029123702976661856\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1052\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1053\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.323224043715847\n", + "T-Statistic: -6.786089292025601\n", + "P-Value: 0.006541270274250083\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.288135593220339\n", + "T-Statistic: -5.089358370900108\n", + "P-Value: 0.014661823332742536\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1053\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1054\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3608606557377049\n", + "T-Statistic: -6.358986733603966\n", + "P-Value: 0.007869248834482912\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3121468926553672\n", + "T-Statistic: -3.4347826086956523\n", + "P-Value: 0.04139322928919692\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1054\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1055\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.32786885245901637\n", + "T-Statistic: -15.380260404150008\n", + "P-Value: 0.0005970483655978608\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.2641242937853107\n", + "T-Statistic: -11.821155866809187\n", + "P-Value: 0.007080250048172652\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1055\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1056\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.44849726775956283\n", + "T-Statistic: -4.876103332248839\n", + "P-Value: 0.016485948236315353\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.40084745762711865\n", + "T-Statistic: -8.576023546581505\n", + "P-Value: 0.003332372210409887\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1056\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1057\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3813296903460838\n", + "T-Statistic: -3.099243212836256\n", + "P-Value: 0.09023900265155212\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3121468926553672\n", + "T-Statistic: -4.822765530016256\n", + "P-Value: 0.016987841571810883\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1057\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1058\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3650956284153006\n", + "T-Statistic: -4.977491357685013\n", + "P-Value: 0.015583918885552142\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3081214689265537\n", + "T-Statistic: -10.058098809188749\n", + "P-Value: 0.002092572636159546\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1058\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1059\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.38545081967213113\n", + "T-Statistic: -2.575853873389877\n", + "P-Value: 0.08207463601397176\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.32561205273069677\n", + "T-Statistic: -3.1284403669724816\n", + "P-Value: 0.08877909183169339\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1059\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1060\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -5.945876360497398\n", + "P-Value: 0.009512113138117317\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.2994350282485876\n", + "T-Statistic: -5.277823574803065\n", + "P-Value: 0.013263065803865479\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1060\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1061\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.37739071038251365\n", + "T-Statistic: -5.80205523533122\n", + "P-Value: 0.010188900334164743\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.2915960451977401\n", + "T-Statistic: -2.792542124181373\n", + "P-Value: 0.06827442570311215\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1061\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1062\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -2.8708139379727218\n", + "P-Value: 0.06400768969988004\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.29237288135593215\n", + "T-Statistic: -3.5351298218230793\n", + "P-Value: 0.07153794820044\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1062\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1063\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4312158469945355\n", + "T-Statistic: -3.2417038925561115\n", + "P-Value: 0.04778912095848229\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.379590395480226\n", + "T-Statistic: -2.6941204566469295\n", + "P-Value: 0.07415454501482839\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1063\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1064\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.36933060109289617\n", + "T-Statistic: -6.847157062045659\n", + "P-Value: 0.006376146501783941\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3334039548022599\n", + "T-Statistic: -3.67018433388411\n", + "P-Value: 0.03499558816381141\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1064\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1065\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.36912568306010923\n", + "T-Statistic: -7.1062884927858425\n", + "P-Value: 0.005733467077920565\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.29971751412429376\n", + "T-Statistic: -3.1708738954340325\n", + "P-Value: 0.050446746960355386\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1065\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1066\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -11.123716102549825\n", + "P-Value: 0.0015567823708010134\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.35014124293785315\n", + "T-Statistic: -8.799712020856967\n", + "P-Value: 0.003091964525888444\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1066\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1067\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4166666666666667\n", + "T-Statistic: -2.859337167599795\n", + "P-Value: 0.06461206589437947\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3644067796610169\n", + "T-Statistic: -3.63823689123835\n", + "P-Value: 0.03578597061066121\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1067\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1068\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.41475409836065574\n", + "T-Statistic: -8.016759438310023\n", + "P-Value: 0.004051975840158878\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3940677966101695\n", + "T-Statistic: -2.6400000000000006\n", + "P-Value: 0.07765595434454427\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1068\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1069\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -3.8415339067926904\n", + "P-Value: 0.03111726238521636\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.346045197740113\n", + "T-Statistic: -4.6989671174421845\n", + "P-Value: 0.01823173060555818\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1069\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1070\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.35273224043715845\n", + "T-Statistic: -8.96185148037276\n", + "P-Value: 0.0029318759099350045\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3163841807909604\n", + "T-Statistic: -9.237604307034024\n", + "P-Value: 0.002683909079797024\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1070\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1071\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.3729508196721312\n", + "T-Statistic: -6.271729950256445\n", + "P-Value: 0.008183201757294712\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3204802259887005\n", + "T-Statistic: -6.160684868447694\n", + "P-Value: 0.008606983305788924\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1071\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1072\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -3.845846875093736\n", + "P-Value: 0.031026938964603432\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.2838983050847458\n", + "T-Statistic: -7.452597186167861\n", + "P-Value: 0.005001392247010432\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1072\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1073\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3443306010928962\n", + "T-Statistic: -6.819969086862238\n", + "P-Value: 0.006448975326342902\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.2701271186440678\n", + "T-Statistic: -5.680855091846446\n", + "P-Value: 0.01080903321709519\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1073\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1074\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -2.753689634807033\n", + "P-Value: 0.0705238707366333\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3165960451977401\n", + "T-Statistic: -4.368550947552518\n", + "P-Value: 0.022184572544257958\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1074\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1075\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3471766848816029\n", + "T-Statistic: -2.090326535903569\n", + "P-Value: 0.1717476912950698\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.32055084745762713\n", + "T-Statistic: -6.009400282160132\n", + "P-Value: 0.009231944143766397\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1075\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1076\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.33333333333333337\n", + "T-Statistic: -6.424891128611615\n", + "P-Value: 0.007642542451366619\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.2641242937853107\n", + "T-Statistic: -3.6019080768824088\n", + "P-Value: 0.06917625132966021\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1076\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1077\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.37745901639344265\n", + "T-Statistic: -5.307134884333113\n", + "P-Value: 0.013061404240156596\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3248587570621469\n", + "T-Statistic: -6.653056282246297\n", + "P-Value: 0.006920964466083356\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1077\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1078\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.38558743169398907\n", + "T-Statistic: -3.5605909993218026\n", + "P-Value: 0.037804245228875336\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.34173728813559323\n", + "T-Statistic: -2.5913248074535873\n", + "P-Value: 0.08098113762773655\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1078\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1079\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.30833333333333335\n", + "T-Statistic: -13.194558267956465\n", + "P-Value: 0.0009405413630643912\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.35451977401129947\n", + "T-Statistic: -2.4666666666666663\n", + "P-Value: 0.09033115673485469\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1079\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1080\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7540983606557377\n", + "Average of Other Ratios: 0.3625\n", + "T-Statistic: -11.206538990050577\n", + "P-Value: 0.001523158238254348\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3712570621468927\n", + "T-Statistic: -2.1556241033270016\n", + "P-Value: 0.12006957075894573\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1080\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1081\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.3416666666666666\n", + "T-Statistic: -2.7938036914857083\n", + "P-Value: 0.06820289228132356\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3207627118644068\n", + "T-Statistic: -3.6925370142833875\n", + "P-Value: 0.03445581989413703\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1081\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1082\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.3652322404371584\n", + "T-Statistic: -6.1214946474517395\n", + "P-Value: 0.008763411957595306\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3080508474576271\n", + "T-Statistic: -6.565459858828986\n", + "P-Value: 0.007186932185349032\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1082\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1083\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.38333333333333336\n", + "T-Statistic: -8.021244634914645\n", + "P-Value: 0.004045425154564468\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.34625706214689267\n", + "T-Statistic: -2.534027979956117\n", + "P-Value: 0.08512323508130125\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1083\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1084\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.423224043715847\n", + "T-Statistic: -3.689180746726386\n", + "P-Value: 0.03453618347086469\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.33340395480225987\n", + "T-Statistic: -6.185153671923991\n", + "P-Value: 0.008511170916897891\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1084\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1085\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.42500000000000004\n", + "T-Statistic: -3.2776582466721216\n", + "P-Value: 0.04650778639185666\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.35861581920903957\n", + "T-Statistic: -6.26717557251908\n", + "P-Value: 0.008200036888372677\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1085\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1086\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -2.5040961733848097\n", + "P-Value: 0.08739061882540976\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.291454802259887\n", + "T-Statistic: -4.773212266552967\n", + "P-Value: 0.01747207617796873\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1086\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1087\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7868852459016393\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -10.75090825425996\n", + "P-Value: 0.0017209617612538612\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3627824858757063\n", + "T-Statistic: -4.484419263456089\n", + "P-Value: 0.020682260794596567\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1087\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1088\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.45833333333333337\n", + "T-Statistic: -9.918032786885249\n", + "P-Value: 0.002180339575209168\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.37097457627118646\n", + "T-Statistic: -5.427900519255732\n", + "P-Value: 0.012271547258470204\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1088\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1089\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7540983606557377\n", + "Average of Other Ratios: 0.3083333333333333\n", + "T-Statistic: -27.93518318279224\n", + "P-Value: 0.0001006969515148238\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.29764595103578156\n", + "T-Statistic: -1.787227246173595\n", + "P-Value: 0.21581013089073905\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1089\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1090\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3562841530054645\n", + "T-Statistic: -3.0165050287504305\n", + "P-Value: 0.05691679534461103\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.32895480225988705\n", + "T-Statistic: -7.500747797147324\n", + "P-Value: 0.0049095659068835425\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1090\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1091\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -2.792392128219332\n", + "P-Value: 0.06828293698877178\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.34173728813559323\n", + "T-Statistic: -9.414301217197753\n", + "P-Value: 0.0025394604898687696\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1091\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1092\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3187841530054645\n", + "T-Statistic: -3.0923105332523986\n", + "P-Value: 0.053616690038148854\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.22768361581920904\n", + "T-Statistic: -2.8518518518518525\n", + "P-Value: 0.06501008999538796\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1092\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1093\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -5.034902560136769\n", + "P-Value: 0.015101540087103106\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.2966101694915254\n", + "T-Statistic: -4.066314336189686\n", + "P-Value: 0.026826320116124475\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1093\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1094\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -6.611700289223251\n", + "P-Value: 0.0070448899240306145\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.32069209039548024\n", + "T-Statistic: -5.480791700836992\n", + "P-Value: 0.011945272816605556\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1094\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1095\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3692622950819672\n", + "T-Statistic: -3.8314981973665483\n", + "P-Value: 0.03132874291483428\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3080508474576271\n", + "T-Statistic: -3.679543437365697\n", + "P-Value: 0.034768279257972504\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1095\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1096\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.35\n", + "T-Statistic: -4.446971954645207\n", + "P-Value: 0.021152969420364667\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.29096045197740117\n", + "T-Statistic: -3.1558437213360135\n", + "P-Value: 0.05103452997518892\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1096\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1097\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4068306010928962\n", + "T-Statistic: -5.623397158172275\n", + "P-Value: 0.011120322780836269\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3584745762711864\n", + "T-Statistic: -3.746821857674737\n", + "P-Value: 0.0331886297757254\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1097\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1098\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.37684426229508194\n", + "T-Statistic: -3.3673616030988653\n", + "P-Value: 0.04349562581061357\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3586158192090395\n", + "T-Statistic: -15.62463762709096\n", + "P-Value: 0.0005697357387431864\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1098\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1099\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.36250000000000004\n", + "T-Statistic: -3.567536931880534\n", + "P-Value: 0.03761787225198254\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.278954802259887\n", + "T-Statistic: -2.9455339690026325\n", + "P-Value: 0.06024089377159606\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1099\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1100\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.33599726775956285\n", + "T-Statistic: -19.586542860762346\n", + "P-Value: 0.0002907615152797266\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.32492937853107345\n", + "T-Statistic: -3.107327943782055\n", + "P-Value: 0.05299174949102144\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1100\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1101\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.75\n", + "Average of Other Ratios: 0.3402322404371585\n", + "T-Statistic: -98.69856142441458\n", + "P-Value: 2.2928614096026574e-06\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.35444915254237286\n", + "T-Statistic: -4.977643693767403\n", + "P-Value: 0.01558261256489159\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1101\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1102\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.34763205828779603\n", + "T-Statistic: -2.164602989862948\n", + "P-Value: 0.16283491133527353\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.314406779661017\n", + "T-Statistic: -1.5149901793251697\n", + "P-Value: 0.2689983409780041\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1102\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1103\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7540983606557377\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -13.34516471643176\n", + "P-Value: 0.0009094713400266177\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.417725988700565\n", + "T-Statistic: -3.7729601826121053\n", + "P-Value: 0.03259975536350789\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1103\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1104\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3486338797814208\n", + "T-Statistic: -4.5541089442634854\n", + "P-Value: 0.019841812100798096\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.253319209039548\n", + "T-Statistic: -2.5734510578167744\n", + "P-Value: 0.0822460968971868\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1104\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1105\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.325\n", + "T-Statistic: -10.30712201881151\n", + "P-Value: 0.0019477541958685205\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.2942090395480226\n", + "T-Statistic: -2.558441558441555\n", + "P-Value: 0.2372076297039761\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1105\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1106\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.3416666666666666\n", + "T-Statistic: -9.384048173269779\n", + "P-Value: 0.0025634488134986455\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.27394067796610166\n", + "T-Statistic: -7.9213555129905915\n", + "P-Value: 0.004194695908330953\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1106\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1107\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3570355191256831\n", + "T-Statistic: -5.484975321134636\n", + "P-Value: 0.011919951523226141\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.325\n", + "T-Statistic: -3.3471216807729713\n", + "P-Value: 0.04415312534526643\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1107\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1108\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4395491803278689\n", + "T-Statistic: -6.233752412361234\n", + "P-Value: 0.00832498749577228\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.40918079096045196\n", + "T-Statistic: -3.9630593303183272\n", + "P-Value: 0.028696120256434428\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1108\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1109\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.33613387978142073\n", + "T-Statistic: -5.360731285926958\n", + "P-Value: 0.012702901609225676\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.2911016949152542\n", + "T-Statistic: -3.4316055779260313\n", + "P-Value: 0.0414893603948339\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1109\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1110\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4563524590163935\n", + "T-Statistic: -3.2403169660324136\n", + "P-Value: 0.04783943753394908\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.400635593220339\n", + "T-Statistic: -8.675158396579372\n", + "P-Value: 0.0032228830687179013\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1110\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1111\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.3875\n", + "T-Statistic: -20.121911614894287\n", + "P-Value: 0.0002682965121865791\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3427495291902072\n", + "T-Statistic: -3.250494467879352\n", + "P-Value: 0.08302849336823244\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1111\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1112\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -9.166095697580069\n", + "P-Value: 0.002745459142872881\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.38008474576271184\n", + "T-Statistic: -3.4264466597407415\n", + "P-Value: 0.04164606550795528\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1112\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1113\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -5.742446164819555\n", + "P-Value: 0.010487900478810956\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.320409604519774\n", + "T-Statistic: -6.489451756285192\n", + "P-Value: 0.007428710958179485\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1113\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1114\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -7.815029537231881\n", + "P-Value: 0.004361685111408271\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.36278248587570616\n", + "T-Statistic: -4.518646959031556\n", + "P-Value: 0.02026382990059579\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1114\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1115\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.44999999999999996\n", + "T-Statistic: -3.1053610315875337\n", + "P-Value: 0.05307307725713686\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.38806497175141247\n", + "T-Statistic: -9.914490233494313\n", + "P-Value: 0.002182622087591197\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1115\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1116\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.36454918032786887\n", + "T-Statistic: -3.2455626872934524\n", + "P-Value: 0.04764947889276525\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.3135593220338983\n", + "T-Statistic: -6.377512576754887\n", + "P-Value: 0.007804638131363213\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1116\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1117\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.37356557377049177\n", + "T-Statistic: -3.757035055298464\n", + "P-Value: 0.032956925528189414\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.2616525423728814\n", + "T-Statistic: -8.372415728614014\n", + "P-Value: 0.0035731576383694914\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1117\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1118\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.39999999999999997\n", + "T-Statistic: -5.493611809608042\n", + "P-Value: 0.011867900939411315\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3500706214689266\n", + "T-Statistic: -2.6979742584262723\n", + "P-Value: 0.07391274673643934\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1118\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1119\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3897540983606558\n", + "T-Statistic: -7.467934583305662\n", + "P-Value: 0.004971898730909863\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3686440677966102\n", + "T-Statistic: -7.629306376694646\n", + "P-Value: 0.004675053414430021\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1119\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1120\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.42342896174863387\n", + "T-Statistic: -4.800324631602227\n", + "P-Value: 0.01720494304786845\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3728813559322034\n", + "T-Statistic: -2.1448905695991947\n", + "P-Value: 0.12129141727935283\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1120\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1121\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.33749999999999997\n", + "T-Statistic: -14.977768942526218\n", + "P-Value: 0.0006459569455958266\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3162429378531073\n", + "T-Statistic: -3.2177938896146965\n", + "P-Value: 0.04866601725501004\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1121\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1122\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.29849726775956287\n", + "T-Statistic: -4.379387604175447\n", + "P-Value: 0.022038198857139507\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.2699152542372881\n", + "T-Statistic: -6.495776752201798\n", + "P-Value: 0.007408187085051301\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1122\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1123\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.39166666666666666\n", + "T-Statistic: -3.42163677485204\n", + "P-Value: 0.041792847188906634\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3432203389830508\n", + "T-Statistic: -3.505866051946874\n", + "P-Value: 0.039314608332933376\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1123\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1124\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -16.463141062057648\n", + "P-Value: 0.0004877463789961118\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3168785310734463\n", + "T-Statistic: -3.110976497704088\n", + "P-Value: 0.052841304640431436\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1124\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1125\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.3688524590163934\n", + "T-Statistic: -7.8974364585461565\n", + "P-Value: 0.00423151561991115\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3041666666666667\n", + "T-Statistic: -4.0423517251725425\n", + "P-Value: 0.027245933023793766\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1125\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1126\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -3.8425267583017693\n", + "P-Value: 0.03109643993412762\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.32627118644067793\n", + "T-Statistic: -1.6531163063339527\n", + "P-Value: 0.24012067051362174\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1126\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1127\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.3916666666666667\n", + "T-Statistic: -4.682079879777176\n", + "P-Value: 0.01841047771126608\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.325894538606403\n", + "T-Statistic: -9.506969760375863\n", + "P-Value: 0.01088379244125519\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1127\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1128\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.4646174863387978\n", + "T-Statistic: -5.6726426012368485\n", + "P-Value: 0.010852821078198268\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3924435028248588\n", + "T-Statistic: -4.386052107630403\n", + "P-Value: 0.021948799914748462\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1128\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1129\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.31666666666666665\n", + "T-Statistic: -3.324802624172649\n", + "P-Value: 0.04489286075866494\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.2489406779661017\n", + "T-Statistic: -3.2234880893826983\n", + "P-Value: 0.04845534939455295\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1129\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1130\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.31666666666666665\n", + "T-Statistic: -5.419860209810303\n", + "P-Value: 0.012322162365242495\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.2966101694915254\n", + "T-Statistic: -6.403332465729597\n", + "P-Value: 0.00771574670699146\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1130\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1131\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.36045081967213116\n", + "T-Statistic: -4.440891852865153\n", + "P-Value: 0.02123070084724967\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -4.837646025991297\n", + "P-Value: 0.016845843964025876\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1131\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1132\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3941256830601093\n", + "T-Statistic: -3.223851145752677\n", + "P-Value: 0.04844195656899026\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3288841807909605\n", + "T-Statistic: -6.129713906579408\n", + "P-Value: 0.0087302972100172\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1132\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1133\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.45833333333333337\n", + "T-Statistic: -6.722810708442813\n", + "P-Value: 0.00671836273975336\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3926553672316384\n", + "T-Statistic: -3.4156502553198673\n", + "P-Value: 0.04197645602545185\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1133\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1134\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.41461748633879786\n", + "T-Statistic: -4.609601498177781\n", + "P-Value: 0.01920401476029972\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3771186440677966\n", + "T-Statistic: -3.6395833778303657\n", + "P-Value: 0.03575220047050311\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1134\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1135\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3482923497267759\n", + "T-Statistic: -3.935890584039202\n", + "P-Value: 0.029215786175393728\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.24879943502824858\n", + "T-Statistic: -5.13286206856859\n", + "P-Value: 0.014322469145621968\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1135\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1136\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.35\n", + "T-Statistic: -14.97130283665096\n", + "P-Value: 0.0006467854175943383\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.25838041431261766\n", + "T-Statistic: -2.478383793597629\n", + "P-Value: 0.13145422697640002\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1136\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1137\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7540983606557377\n", + "Average of Other Ratios: 0.3541666666666667\n", + "T-Statistic: -16.708601239682135\n", + "P-Value: 0.00046674362746457226\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3209745762711865\n", + "T-Statistic: -3.005502067896227\n", + "P-Value: 0.05741679214585345\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1137\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1138\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -2.2740071535885997\n", + "P-Value: 0.10752712396614864\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.29103107344632767\n", + "T-Statistic: -3.1775375076563694\n", + "P-Value: 0.05018888898927705\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1138\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1139\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.43907103825136606\n", + "T-Statistic: -2.820281518337947\n", + "P-Value: 0.06672285377972131\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.38411016949152543\n", + "T-Statistic: -2.4611692476881775\n", + "P-Value: 0.09077318658872167\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1139\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1140\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.3625\n", + "T-Statistic: -3.419275812902317\n", + "P-Value: 0.04186513660401589\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3458333333333333\n", + "T-Statistic: -4.644284279214491\n", + "P-Value: 0.018818859918736594\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1140\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1141\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -6.456069857786826\n", + "P-Value: 0.007538278964860644\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.32055084745762713\n", + "T-Statistic: -4.426845916974451\n", + "P-Value: 0.021411691624411183\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1141\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1142\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.4291666666666667\n", + "T-Statistic: -1.8998904386836808\n", + "P-Value: 0.15364813434857005\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.37153954802259886\n", + "T-Statistic: -3.1115472159109827\n", + "P-Value: 0.052817820285826836\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1142\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1143\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.44999999999999996\n", + "T-Statistic: -3.585966147227154\n", + "P-Value: 0.03712903167578977\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.4094632768361582\n", + "T-Statistic: -3.037748906538044\n", + "P-Value: 0.055966806959142974\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1143\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1144\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.4291666666666667\n", + "T-Statistic: -4.10299965843394\n", + "P-Value: 0.02619990938534542\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.40494350282485875\n", + "T-Statistic: -3.713511479606386\n", + "P-Value: 0.03395898358756934\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1144\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1145\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -5.466397186258622\n", + "P-Value: 0.01203293462437204\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.36864406779661013\n", + "T-Statistic: -5.628616053819864\n", + "P-Value: 0.011091567639778466\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1145\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1146\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3855191256830601\n", + "T-Statistic: -3.236738871470941\n", + "P-Value: 0.047969557784140165\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3547316384180791\n", + "T-Statistic: -2.9144455089837993\n", + "P-Value: 0.061773504814927487\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1146\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1147\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.38982240437158466\n", + "T-Statistic: -7.304853191112693\n", + "P-Value: 0.005297682541612248\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3204802259887006\n", + "T-Statistic: -7.80733944954129\n", + "P-Value: 0.0043741006479608106\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1147\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1148\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8360655737704918\n", + "Average of Other Ratios: 0.34583333333333327\n", + "T-Statistic: -17.804840632796406\n", + "P-Value: 0.00038631936941714036\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3813559322033898\n", + "T-Statistic: -3.1047293380092054\n", + "P-Value: 0.05309922984140076\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1148\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1149\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.3708333333333333\n", + "T-Statistic: -5.272729644436734\n", + "P-Value: 0.01329852584746764\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.35444915254237286\n", + "T-Statistic: -2.8491361177963865\n", + "P-Value: 0.06515525258058204\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1149\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1150\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3483606557377049\n", + "T-Statistic: -4.11223200027819\n", + "P-Value: 0.026045241625233456\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.26560734463276836\n", + "T-Statistic: -6.047006896155811\n", + "P-Value: 0.009071170018542232\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1150\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1151\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.38750000000000007\n", + "T-Statistic: -4.357648863022137\n", + "P-Value: 0.022333101933145045\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.35000000000000003\n", + "T-Statistic: -2.2152876697379726\n", + "P-Value: 0.11354046971526367\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1151\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1152\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.32916666666666666\n", + "T-Statistic: -6.767511177347243\n", + "P-Value: 0.006592619506710128\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3347457627118644\n", + "T-Statistic: -4.554467869037332\n", + "P-Value: 0.019837599235630846\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1152\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1153\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4186475409836065\n", + "T-Statistic: -2.434445731879166\n", + "P-Value: 0.0929600077713279\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.34180790960451973\n", + "T-Statistic: -6.35085296108589\n", + "P-Value: 0.007897838244939661\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1153\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1154\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.36666666666666664\n", + "T-Statistic: -5.1238506750677635\n", + "P-Value: 0.0143919153374869\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.33050847457627114\n", + "T-Statistic: -2.733333333333336\n", + "P-Value: 0.07173922515901705\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1154\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1155\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.41072404371584703\n", + "T-Statistic: -10.34274767901109\n", + "P-Value: 0.0019281339723313756\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.346045197740113\n", + "T-Statistic: -13.339459376998303\n", + "P-Value: 0.0009106232895004249\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1155\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1156\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.35280054644808745\n", + "T-Statistic: -9.808946709611767\n", + "P-Value: 0.0022521026539224693\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.2702683615819209\n", + "T-Statistic: -5.290392517155741\n", + "P-Value: 0.013176097120409095\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1156\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1157\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.43531420765027323\n", + "T-Statistic: -2.293238524273031\n", + "P-Value: 0.10564172479014122\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3712570621468927\n", + "T-Statistic: -4.1945951901363205\n", + "P-Value: 0.024716200055892694\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1157\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1158\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.35273224043715845\n", + "T-Statistic: -8.189147517586767\n", + "P-Value: 0.0038099469501537706\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3499293785310735\n", + "T-Statistic: -3.9622923434347794\n", + "P-Value: 0.02871062689188144\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1158\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1159\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.36898907103825135\n", + "T-Statistic: -9.328181302832693\n", + "P-Value: 0.0026085416180980188\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.3305084745762712\n", + "T-Statistic: -3.3049457887636606\n", + "P-Value: 0.045564249549415244\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1159\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1160\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -3.3240845596280026\n", + "P-Value: 0.04491692010565002\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.326271186440678\n", + "T-Statistic: -5.094198380791346\n", + "P-Value: 0.014623553591468103\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1160\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1161\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.38162568306010936\n", + "T-Statistic: -5.53218920098128\n", + "P-Value: 0.011638997397509718\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3377824858757062\n", + "T-Statistic: -5.297341314999525\n", + "P-Value: 0.013128335633175506\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1161\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1162\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4438524590163935\n", + "T-Statistic: -4.609409203433426\n", + "P-Value: 0.01920617859490499\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3799435028248587\n", + "T-Statistic: -4.13405315401556\n", + "P-Value: 0.02568432777343715\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1162\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1163\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.35416666666666663\n", + "T-Statistic: -3.463054126930755\n", + "P-Value: 0.040550141822148895\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.34625706214689267\n", + "T-Statistic: -3.451648730864812\n", + "P-Value: 0.040887609276162\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1163\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1164\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3030054644808743\n", + "T-Statistic: -3.9665722337998846\n", + "P-Value: 0.028629798248878405\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.25727401129943506\n", + "T-Statistic: -3.278863962441917\n", + "P-Value: 0.0464655758073765\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1164\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1165\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.37336065573770494\n", + "T-Statistic: -8.550441690064975\n", + "P-Value: 0.003361422362271298\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.34745762711864403\n", + "T-Statistic: -4.256851455445149\n", + "P-Value: 0.02376922627056341\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1165\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1166\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -4.685055463096139\n", + "P-Value: 0.01837881723814625\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3163841807909604\n", + "T-Statistic: -3.333333333333336\n", + "P-Value: 0.044608278994442765\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1166\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1167\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.31127049180327865\n", + "T-Statistic: -9.651006540622726\n", + "P-Value: 0.002361656973312143\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.30783898305084745\n", + "T-Statistic: -3.2916992909941314\n", + "P-Value: 0.04601922125686048\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1167\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1168\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.38599726775956283\n", + "T-Statistic: -15.465401308907632\n", + "P-Value: 0.0005873388348340652\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3500706214689266\n", + "T-Statistic: -3.1298749223282756\n", + "P-Value: 0.052070602215977944\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1168\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1169\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.2568306010928962\n", + "T-Statistic: -10.419231568279397\n", + "P-Value: 0.0018868885475834876\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.21920903954802262\n", + "T-Statistic: -2.9631204814438217\n", + "P-Value: 0.09751860335760698\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1169\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1170\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8032786885245902\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -8.376268479684917\n", + "P-Value: 0.0035683913912364033\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3543785310734463\n", + "T-Statistic: -6.117934775079367\n", + "P-Value: 0.008777805575829042\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1170\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1171\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3978825136612022\n", + "T-Statistic: -2.035753205981991\n", + "P-Value: 0.1345958069792505\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.35021186440677965\n", + "T-Statistic: -5.6736454811631605\n", + "P-Value: 0.010847461425876656\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1171\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1172\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3980874316939891\n", + "T-Statistic: -7.600738322488857\n", + "P-Value: 0.004725867063471558\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3694915254237288\n", + "T-Statistic: -1.777777777777776\n", + "P-Value: 0.32619726158657\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1172\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1173\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7868852459016393\n", + "Average of Other Ratios: 0.39166666666666666\n", + "T-Statistic: -27.381546373206323\n", + "P-Value: 0.00010690928977807902\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3669491525423729\n", + "T-Statistic: -3.4506787823167455\n", + "P-Value: 0.040916473140808284\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1173\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1174\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.3278005464480874\n", + "T-Statistic: -5.343021351720979\n", + "P-Value: 0.01281992383399354\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.2923728813559322\n", + "T-Statistic: -6.160404875922859\n", + "P-Value: 0.00860808784608318\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1174\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1175\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.41857923497267757\n", + "T-Statistic: -3.3484587546107205\n", + "P-Value: 0.0441093025667818\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.32090395480225986\n", + "T-Statistic: -7.2901037265115765\n", + "P-Value: 0.0053285197621824095\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1175\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1176\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.33989071038251367\n", + "T-Statistic: -4.377554579883222\n", + "P-Value: 0.02206287000445764\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.286864406779661\n", + "T-Statistic: -3.6376865077145064\n", + "P-Value: 0.03579978607225466\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1176\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1177\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.39207650273224043\n", + "T-Statistic: -6.083165423794337\n", + "P-Value: 0.025975157859805906\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.39237288135593223\n", + "T-Statistic: -3.433070549496972\n", + "P-Value: 0.04144499780826165\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1177\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1178\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.3208333333333333\n", + "T-Statistic: -15.854306407643215\n", + "P-Value: 0.0005455614883016697\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3372881355932204\n", + "T-Statistic: -3.3003043252568145\n", + "P-Value: 0.08083454343693727\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1178\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1179\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.28189890710382515\n", + "T-Statistic: -8.369667988980005\n", + "P-Value: 0.003576562016483774\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.2921845574387947\n", + "T-Statistic: -3.052528420785673\n", + "P-Value: 0.09264712636043493\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1179\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1180\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3903005464480874\n", + "T-Statistic: -6.562635068330882\n", + "P-Value: 0.0071957310516421215\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.37966101694915255\n", + "T-Statistic: -4.431293675255974\n", + "P-Value: 0.02135416398761436\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1180\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1181\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.3730191256830601\n", + "T-Statistic: -6.867274869817217\n", + "P-Value: 0.006322952397605397\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -3.3728798057819063\n", + "P-Value: 0.043318525364938\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1181\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1182\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.32568306010928966\n", + "T-Statistic: -4.191489361702128\n", + "P-Value: 0.052479362028526676\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.295409604519774\n", + "T-Statistic: -5.351131248404091\n", + "P-Value: 0.012766161878428656\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1182\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1183\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.35000000000000003\n", + "T-Statistic: -6.5653506061794005\n", + "P-Value: 0.0071872722304183664\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3373587570621469\n", + "T-Statistic: -4.004253706903789\n", + "P-Value: 0.02793063892749739\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1183\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1184\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.3524590163934426\n", + "T-Statistic: -10.447778102751764\n", + "P-Value: 0.0018717942236663328\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3082627118644068\n", + "T-Statistic: -3.4319134544954104\n", + "P-Value: 0.04148003219890615\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1184\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1185\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.36249999999999993\n", + "T-Statistic: -70.37704918032789\n", + "P-Value: 6.3221094755592405e-06\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.34187853107344635\n", + "T-Statistic: -4.233170395077674\n", + "P-Value: 0.024123819842227313\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1185\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1186\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.45621584699453555\n", + "T-Statistic: -2.481511075867372\n", + "P-Value: 0.0891506613942868\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3927966101694915\n", + "T-Statistic: -2.484400110285176\n", + "P-Value: 0.08892311315035355\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1186\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1187\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4271174863387978\n", + "T-Statistic: -3.4482809420111318\n", + "P-Value: 0.040987939768970645\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3501412429378531\n", + "T-Statistic: -6.005437958250537\n", + "P-Value: 0.009249100739188856\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1187\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1188\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -23.226517386743268\n", + "P-Value: 0.000174834804970807\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.3246468926553672\n", + "T-Statistic: -8.378851501317461\n", + "P-Value: 0.003565200635279848\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1188\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1189\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.39583333333333337\n", + "T-Statistic: -7.677595628415292\n", + "P-Value: 0.004590789402025528\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3753531073446328\n", + "T-Statistic: -3.92529890966906\n", + "P-Value: 0.029421652663864435\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1189\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1190\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3941256830601093\n", + "T-Statistic: -3.7613782647305922\n", + "P-Value: 0.03285902039196703\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3626412429378531\n", + "T-Statistic: -4.119009202618319\n", + "P-Value: 0.02593245296220258\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1190\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1191\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.3416666666666666\n", + "T-Statistic: -13.4125186386734\n", + "P-Value: 0.0008960173086668277\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.326271186440678\n", + "T-Statistic: -2.5654413105709444\n", + "P-Value: 0.08282085362511153\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1191\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1192\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3273224043715847\n", + "T-Statistic: -3.665214293431724\n", + "P-Value: 0.0351170707372584\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.2235169491525424\n", + "T-Statistic: -2.8217002208427564\n", + "P-Value: 0.06664468710770881\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1192\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1193\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.52698087431694\n", + "T-Statistic: -2.6670409685182177\n", + "P-Value: 0.07588150981935955\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.4976694915254237\n", + "T-Statistic: -2.725345181640537\n", + "P-Value: 0.0722232481118407\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1193\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1194\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -22.125683060109264\n", + "P-Value: 0.00020211389888430634\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3484934086629002\n", + "T-Statistic: -3.5009766627567247\n", + "P-Value: 0.07279105093612302\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1194\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1195\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3484289617486339\n", + "T-Statistic: -2.8306761052244935\n", + "P-Value: 0.06615277677623232\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.3038135593220339\n", + "T-Statistic: -4.999032554530682\n", + "P-Value: 0.015400604884304166\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1195\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1196\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3771857923497267\n", + "T-Statistic: -2.618745744139447\n", + "P-Value: 0.07908677590411582\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3333333333333333\n", + "Average of Other Ratios: 0.30508474576271183\n", + "T-Statistic: -4.082482904638637\n", + "P-Value: 0.02654788546719936\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1196\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1197\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3983606557377049\n", + "T-Statistic: -3.9828547448399347\n", + "P-Value: 0.02832495388510772\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3290960451977401\n", + "T-Statistic: -4.253217026259508\n", + "P-Value: 0.023823207735453863\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1197\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1198\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.3708333333333333\n", + "T-Statistic: -2.3515482695810577\n", + "P-Value: 0.10016489140788615\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3121468926553672\n", + "T-Statistic: -3.9090909090909123\n", + "P-Value: 0.02974030553139564\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1198\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1199\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3691256830601093\n", + "T-Statistic: -7.521674420991118\n", + "P-Value: 0.004870350805489641\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3165960451977401\n", + "T-Statistic: -3.096482698963895\n", + "P-Value: 0.05344214083896168\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1199\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1200\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3650956284153005\n", + "T-Statistic: -41.98171043120708\n", + "P-Value: 2.974431577617351e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3545197740112994\n", + "T-Statistic: -6.4902085496187105\n", + "P-Value: 0.007426251317674435\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1200\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1201\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3980191256830601\n", + "T-Statistic: -3.2824011075555344\n", + "P-Value: 0.04634202450941883\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3628531073446328\n", + "T-Statistic: -2.838043708162847\n", + "P-Value: 0.06575238490643358\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1201\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1202\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.39344262295081966\n", + "Average of Other Ratios: 0.35416666666666663\n", + "T-Statistic: -3.142076502732241\n", + "P-Value: 0.051580525827867194\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3163841807909605\n", + "T-Statistic: -4.003203845127177\n", + "P-Value: 0.027949819151750765\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1202\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1203\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -11.78086978660089\n", + "P-Value: 0.0013145795310678767\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.33354519774011293\n", + "T-Statistic: -2.8433656242933654\n", + "P-Value: 0.06546504459353619\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1203\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1204\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.38333333333333336\n", + "T-Statistic: -8.021840834286337\n", + "P-Value: 0.004044555451710541\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.37966101694915255\n", + "T-Statistic: -4.5490523794544675\n", + "P-Value: 0.01990128684164212\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1204\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1205\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.332103825136612\n", + "T-Statistic: -15.850883671738368\n", + "P-Value: 0.0005459116401288157\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.30783898305084745\n", + "T-Statistic: -2.3170196256606355\n", + "P-Value: 0.10336527760682507\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1205\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1206\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.37329234972677594\n", + "T-Statistic: -3.6587848750156016\n", + "P-Value: 0.03527502614042613\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3293079096045198\n", + "T-Statistic: -7.7130494574412145\n", + "P-Value: 0.004530195336094142\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1206\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1207\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.29863387978142075\n", + "T-Statistic: -2.4701740765596973\n", + "P-Value: 0.09005051324793042\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.22344632768361583\n", + "T-Statistic: -3.080575323621966\n", + "P-Value: 0.05411151529897513\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1207\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1208\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -8.810214059275827\n", + "P-Value: 0.0030812498509113165\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.30357815442561203\n", + "T-Statistic: -3.206327062041345\n", + "P-Value: 0.08504610921846024\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1208\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1209\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.4105874316939891\n", + "T-Statistic: -1.5057124818059264\n", + "P-Value: 0.22921714041813998\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.32902542372881355\n", + "T-Statistic: -4.458405259144846\n", + "P-Value: 0.02100779526554658\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1209\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1210\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.22083333333333333\n", + "T-Statistic: -6.909375426419404\n", + "P-Value: 0.0062135098686104855\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.14752824858757063\n", + "T-Statistic: -6.30893579014757\n", + "P-Value: 0.008047357424001828\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1210\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1211\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.41509562841530057\n", + "T-Statistic: -6.219125997443245\n", + "P-Value: 0.00838045426704938\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.35021186440677965\n", + "T-Statistic: -2.551953540367511\n", + "P-Value: 0.08379991236656559\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1211\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1212\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.38995901639344266\n", + "T-Statistic: -5.020631036144959\n", + "P-Value: 0.015219613947787137\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3418079096045198\n", + "T-Statistic: -4.389381125701737\n", + "P-Value: 0.02190431958759874\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1212\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1213\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.3041666666666667\n", + "T-Statistic: -5.783528726099863\n", + "P-Value: 0.010280619668641678\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.21532485875706214\n", + "T-Statistic: -4.8493624334243925\n", + "P-Value: 0.016735124561985816\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1213\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1214\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.34378415300546444\n", + "T-Statistic: -3.106070198750089\n", + "P-Value: 0.05304373654077345\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.2995056497175141\n", + "T-Statistic: -5.3217343632850165\n", + "P-Value: 0.012962453260928666\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1214\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1215\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.34433060109289615\n", + "T-Statistic: -4.489204827567853\n", + "P-Value: 0.020623088486061487\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3432203389830508\n", + "T-Statistic: -2.896998294604734\n", + "P-Value: 0.06265501679556046\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1215\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1216\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.29166666666666663\n", + "T-Statistic: -15.55542491509537\n", + "P-Value: 0.000577299561388648\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.21532485875706214\n", + "T-Statistic: -3.1169524568281517\n", + "P-Value: 0.05259605209854495\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1216\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1217\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3945355191256831\n", + "T-Statistic: -2.804910878658928\n", + "P-Value: 0.0675771021782272\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3333333333333333\n", + "T-Statistic: -3.022438607339303\n", + "P-Value: 0.05664943058417436\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1217\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1218\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3105191256830601\n", + "T-Statistic: -4.772347925976209\n", + "P-Value: 0.017480680770223765\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.26984463276836157\n", + "T-Statistic: -4.292077854283091\n", + "P-Value: 0.023254116236581038\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1218\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1219\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.3375\n", + "T-Statistic: -19.209836065573764\n", + "P-Value: 0.0003080910681863436\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.30388418079096047\n", + "T-Statistic: -3.079787687349886\n", + "P-Value: 0.054144932145154395\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1219\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1220\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.3833333333333333\n", + "T-Statistic: -4.318708814548055\n", + "P-Value: 0.022874240057608075\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.32561205273069677\n", + "T-Statistic: -1.3195730536286363\n", + "P-Value: 0.31778111693939937\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1220\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1221\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3239754098360656\n", + "T-Statistic: -5.08778567271395\n", + "P-Value: 0.014674286692950959\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.2699152542372881\n", + "T-Statistic: -2.3318495798744\n", + "P-Value: 0.10197574453300426\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1221\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1222\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3812158469945355\n", + "T-Statistic: -2.9098120190283097\n", + "P-Value: 0.06200608865891807\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3389830508474576\n", + "T-Statistic: -3.119096328227442\n", + "P-Value: 0.05250841765245064\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1222\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1223\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.2569672131147541\n", + "T-Statistic: -3.7827690734466466\n", + "P-Value: 0.03238222739381364\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.1352401129943503\n", + "T-Statistic: -8.430881102204697\n", + "P-Value: 0.003501725063291807\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1223\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1224\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.2875\n", + "T-Statistic: -16.3510804808083\n", + "P-Value: 0.0004977534397666609\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.2489406779661017\n", + "T-Statistic: -7.020021492444736\n", + "P-Value: 0.005937531487381006\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1224\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1225\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.41250000000000003\n", + "T-Statistic: -1.9095681132275377\n", + "P-Value: 0.15219065910307153\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.328954802259887\n", + "T-Statistic: -3.1151717210383336\n", + "P-Value: 0.05266898293338368\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1225\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1226\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -4.560818191534335\n", + "P-Value: 0.019763254051599725\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3122881355932204\n", + "T-Statistic: -3.467950511615431\n", + "P-Value: 0.04040635351163493\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1226\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1227\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.35416666666666663\n", + "T-Statistic: -13.929800773742766\n", + "P-Value: 0.0008010077875496463\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3260828625235405\n", + "T-Statistic: -3.6801457918777314\n", + "P-Value: 0.0665501469345416\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1227\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1228\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -7.498639437160465\n", + "P-Value: 0.004913539903160214\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.354590395480226\n", + "T-Statistic: -5.215051241964461\n", + "P-Value: 0.01370881147200009\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1228\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1229\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.36919398907103823\n", + "T-Statistic: -2.8757941403931446\n", + "P-Value: 0.06374762528542408\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3038135593220339\n", + "T-Statistic: -9.817377185403636\n", + "P-Value: 0.002246445968285984\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1229\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1230\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.28169398907103826\n", + "T-Statistic: -3.4441072444884826\n", + "P-Value: 0.04111271435550431\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.26596045197740115\n", + "T-Statistic: -2.465458081419365\n", + "P-Value: 0.0904281089259112\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1230\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1231\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.36133879781420764\n", + "T-Statistic: -10.501771543064775\n", + "P-Value: 0.0018436795460142676\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.36264124293785316\n", + "T-Statistic: -5.251461666642595\n", + "P-Value: 0.0134479218636803\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1231\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1232\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.32916666666666666\n", + "T-Statistic: -20.7021431957843\n", + "P-Value: 0.00024648403432591493\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.27443502824858756\n", + "T-Statistic: -3.458769578971492\n", + "P-Value: 0.04067649752950825\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1232\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1233\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.3896857923497268\n", + "T-Statistic: -7.435895472354905\n", + "P-Value: 0.005033772413915748\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.34173728813559323\n", + "T-Statistic: -3.108570507260323\n", + "P-Value: 0.052940452969860205\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1233\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1234\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.3541666666666667\n", + "T-Statistic: -5.824341929189313\n", + "P-Value: 0.01007998096990893\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3220338983050847\n", + "T-Statistic: -3.55910413197229\n", + "P-Value: 0.037844293787820955\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1234\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1235\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.37752732240437153\n", + "T-Statistic: -6.922734884968511\n", + "P-Value: 0.0061793032033680305\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.31652542372881354\n", + "T-Statistic: -2.9875904328450478\n", + "P-Value: 0.058242586475556184\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1235\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1236\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.41666666666666663\n", + "T-Statistic: -9.080332982450201\n", + "P-Value: 0.002821767606719206\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4096045197740113\n", + "T-Statistic: -2.5245779797628782\n", + "P-Value: 0.0858312019877726\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1236\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1237\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.38155737704918036\n", + "T-Statistic: -3.5810708052340003\n", + "P-Value: 0.03725808735441134\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.3089453860640301\n", + "T-Statistic: -2.8553936637391883\n", + "P-Value: 0.10388662348248497\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1237\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1238\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -4.242857392252246\n", + "P-Value: 0.02397794903308461\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3458333333333333\n", + "T-Statistic: -2.306948594944211\n", + "P-Value: 0.10432200362127822\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1238\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1239\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.3625\n", + "T-Statistic: -6.133284818784408\n", + "P-Value: 0.008715961470550591\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3038135593220339\n", + "T-Statistic: -7.137099922436499\n", + "P-Value: 0.005662825302674034\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1239\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1240\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.29494535519125686\n", + "T-Statistic: -3.242117432574401\n", + "P-Value: 0.04777413102072636\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.18559322033898304\n", + "T-Statistic: -10.368238367141146\n", + "P-Value: 0.0019142559726229866\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1240\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1241\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.38749999999999996\n", + "T-Statistic: -2.474836034730356\n", + "P-Value: 0.08967913937363099\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.2786016949152542\n", + "T-Statistic: -6.156258728408069\n", + "P-Value: 0.008624465754537623\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1241\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1242\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.37363387978142076\n", + "T-Statistic: -3.3164206595199035\n", + "P-Value: 0.04517472690275175\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.2956920903954802\n", + "T-Statistic: -2.834146743848625\n", + "P-Value: 0.06596378750306289\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1242\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1243\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.4066256830601093\n", + "T-Statistic: -18.30718715197799\n", + "P-Value: 0.0003555990078420392\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.34187853107344635\n", + "T-Statistic: -3.891797498256559\n", + "P-Value: 0.030085204803105746\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1243\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1244\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.3375\n", + "T-Statistic: -11.473067915690864\n", + "P-Value: 0.0014212819594170815\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.3389830508474576\n", + "T-Statistic: -2.266099806569284\n", + "P-Value: 0.1083141230940181\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1244\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1245\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.31058743169398906\n", + "T-Statistic: -4.4611858896576715\n", + "P-Value: 0.020972683531277587\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.25741525423728817\n", + "T-Statistic: -2.4557675307968445\n", + "P-Value: 0.09121010163141306\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1245\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1246\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4523224043715847\n", + "T-Statistic: -3.6924428701682177\n", + "P-Value: 0.034458070848123835\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.39258474576271185\n", + "T-Statistic: -3.5522131403755797\n", + "P-Value: 0.03803061001063327\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1246\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1247\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8524590163934426\n", + "Average of Other Ratios: 0.3541666666666667\n", + "T-Statistic: -20.02461614309747\n", + "P-Value: 0.00027220292398855736\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.31207627118644066\n", + "T-Statistic: -5.440282569102571\n", + "P-Value: 0.012194130448693096\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1247\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1248\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -15.731996671646355\n", + "P-Value: 0.0005582617929085287\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.32627118644067793\n", + "T-Statistic: -3.788162541206027\n", + "P-Value: 0.032263411126476765\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1248\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1249\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4019808743169399\n", + "T-Statistic: -3.3737593587415304\n", + "P-Value: 0.04329038177941591\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.34745762711864403\n", + "T-Statistic: -3.2791258997889505\n", + "P-Value: 0.046456412128927795\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1249\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1250\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.36092896174863387\n", + "T-Statistic: -4.924844065432519\n", + "P-Value: 0.016044059231596925\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.25741525423728817\n", + "T-Statistic: -5.952990444986046\n", + "P-Value: 0.009480187104661204\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1250\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1251\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -4.511917593907792\n", + "P-Value: 0.020345223247711267\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.3258003766478343\n", + "T-Statistic: -3.4999999999999902\n", + "P-Value: 0.07282735005446969\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1251\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1252\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.37759562841530053\n", + "T-Statistic: -3.483025090320139\n", + "P-Value: 0.039967726509492464\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.2783898305084746\n", + "T-Statistic: -3.9414887073599294\n", + "P-Value: 0.029107725320715257\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1252\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1253\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -14.9760677256279\n", + "P-Value: 0.0006461747777244088\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.364406779661017\n", + "T-Statistic: -2.193378465041791\n", + "P-Value: 0.1158873271135981\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1253\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1254\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.35\n", + "T-Statistic: -4.022062624190254\n", + "P-Value: 0.027607848701928988\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.28255649717514125\n", + "T-Statistic: -3.1504339973019846\n", + "P-Value: 0.05124820108543329\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1254\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1255\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.47691256830601103\n", + "T-Statistic: -3.022304945858794\n", + "P-Value: 0.0566554359290112\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.41377118644067795\n", + "T-Statistic: -2.677569682825962\n", + "P-Value: 0.07520419613663175\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1255\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1256\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3237704918032787\n", + "T-Statistic: -4.585573326772214\n", + "P-Value: 0.019476865011623285\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.24491525423728813\n", + "T-Statistic: -6.057634803744621\n", + "P-Value: 0.00902640078253155\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1256\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1257\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -6.896820529493841\n", + "P-Value: 0.006245884135338378\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.29117231638418084\n", + "T-Statistic: -4.3575806009385145\n", + "P-Value: 0.022334035979091265\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1257\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1258\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.38749999999999996\n", + "T-Statistic: -2.4730386527896364\n", + "P-Value: 0.0898220971946934\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.32916666666666666\n", + "T-Statistic: -10.509267049042842\n", + "P-Value: 0.001839820961868861\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1258\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1259\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.373155737704918\n", + "T-Statistic: -3.7564520938598576\n", + "P-Value: 0.03297009515071729\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.2966101694915254\n", + "T-Statistic: -4.298859857965993\n", + "P-Value: 0.023156605272933835\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1259\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1260\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3278005464480874\n", + "T-Statistic: -5.076865217882905\n", + "P-Value: 0.014761211511120415\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.2870056497175142\n", + "T-Statistic: -17.37458743854924\n", + "P-Value: 0.0004155011309011471\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1260\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1261\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.31086065573770494\n", + "T-Statistic: -2.446306172576996\n", + "P-Value: 0.0919815932829521\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.2303201506591337\n", + "T-Statistic: -5.066892497031914\n", + "P-Value: 0.0368133974917626\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1261\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1262\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -7.2243808310002775\n", + "P-Value: 0.005468843291212087\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.2956920903954802\n", + "T-Statistic: -5.945744266829731\n", + "P-Value: 0.009512707270210679\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1262\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1263\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.35252732240437157\n", + "T-Statistic: -4.378773516969542\n", + "P-Value: 0.022046460036578352\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.2750470809792844\n", + "T-Statistic: -2.0432809561528478\n", + "P-Value: 0.1777389669818575\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1263\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1264\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.37329234972677594\n", + "T-Statistic: -2.5720050130375935\n", + "P-Value: 0.08234949684139341\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3129943502824859\n", + "T-Statistic: -2.399133282225983\n", + "P-Value: 0.09594924200933898\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1264\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1265\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.36666666666666664\n", + "T-Statistic: -5.484666087385246\n", + "P-Value: 0.011921820751493945\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3050847457627119\n", + "T-Statistic: -4.8621138936836426\n", + "P-Value: 0.016615697449670994\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1265\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1266\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.3280054644808743\n", + "T-Statistic: -14.159199715025988\n", + "P-Value: 0.0007631512846613169\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.2699858757062147\n", + "T-Statistic: -3.0417250416423958\n", + "P-Value: 0.055791220683953764\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1266\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1267\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.35416666666666663\n", + "T-Statistic: -6.623110492647877\n", + "P-Value: 0.007010408974038976\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.30348399246704333\n", + "T-Statistic: -4.849951400603131\n", + "P-Value: 0.039981081861906576\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1267\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1268\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -4.326302046694481\n", + "P-Value: 0.022767398907893536\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3093220338983051\n", + "T-Statistic: -4.068509688367149\n", + "P-Value: 0.02678829448291478\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1268\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1269\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.332103825136612\n", + "T-Statistic: -10.547062196561422\n", + "P-Value: 0.0018205268185872753\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.2828389830508474\n", + "T-Statistic: -12.090909090909088\n", + "P-Value: 0.0012175905817395098\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1269\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1270\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.32916666666666666\n", + "T-Statistic: -4.41253184635759\n", + "P-Value: 0.021598198496940495\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.28276836158192087\n", + "T-Statistic: -4.398585621119437\n", + "P-Value: 0.021781940473571012\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1270\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1271\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.29166666666666663\n", + "T-Statistic: -26.924962031836227\n", + "P-Value: 0.00011242244042797529\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.2784604519774011\n", + "T-Statistic: -3.6540488693517794\n", + "P-Value: 0.03539195989892865\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1271\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1272\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3239071038251366\n", + "T-Statistic: -6.101922495764141\n", + "P-Value: 0.008842933069258837\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.2754237288135593\n", + "T-Statistic: -3.195601372491276\n", + "P-Value: 0.049498206131116385\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1272\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1273\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.36448087431693993\n", + "T-Statistic: -5.276046667403815\n", + "P-Value: 0.013275421234137245\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.30357815442561203\n", + "T-Statistic: -3.206327062041345\n", + "P-Value: 0.08504610921846024\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1273\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1274\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3691256830601093\n", + "T-Statistic: -2.611497078469241\n", + "P-Value: 0.07958218244247245\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3347457627118644\n", + "T-Statistic: -2.6807959017177474\n", + "P-Value: 0.07499815239256642\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1274\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1275\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.3605191256830601\n", + "T-Statistic: -7.086477957097316\n", + "P-Value: 0.0057795013156923015\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.35430790960451974\n", + "T-Statistic: -2.9187879140660207\n", + "P-Value: 0.06155652231610408\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1275\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1276\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.3151639344262295\n", + "T-Statistic: -6.220646342838296\n", + "P-Value: 0.008374666202957234\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.24901129943502825\n", + "T-Statistic: -2.864760573339222\n", + "P-Value: 0.06432558093621543\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1276\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1277\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4125\n", + "T-Statistic: -4.719504703575856\n", + "P-Value: 0.01801737147532206\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.3375\n", + "T-Statistic: -8.30959245705665\n", + "P-Value: 0.0036520779681317537\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1277\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1278\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.30000000000000004\n", + "T-Statistic: -4.334146739196724\n", + "P-Value: 0.022657696393709254\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.22351694915254236\n", + "T-Statistic: -3.3098123366183496\n", + "P-Value: 0.04539853318110197\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1278\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1279\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -2.990713926985726\n", + "P-Value: 0.05809751373410958\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3161723163841808\n", + "T-Statistic: -2.483483730018013\n", + "P-Value: 0.0889952126349582\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1279\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1280\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.30416666666666664\n", + "T-Statistic: -5.440801457194902\n", + "P-Value: 0.012190900140945336\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.31666666666666665\n", + "Average of Other Ratios: 0.2754237288135593\n", + "T-Statistic: -3.8676344826299247\n", + "P-Value: 0.030575765124777585\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1280\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1281\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3399590163934426\n", + "T-Statistic: -4.334577337946331\n", + "P-Value: 0.022651694585833295\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.2966101694915254\n", + "T-Statistic: -3.6373066958946425\n", + "P-Value: 0.03580932390070352\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1281\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1282\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3108606557377049\n", + "T-Statistic: -2.3956053387350558\n", + "P-Value: 0.0962542828406371\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3\n", + "Average of Other Ratios: 0.2584745762711864\n", + "T-Statistic: -2.869146214468688\n", + "P-Value: 0.06409507369957781\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1282\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1283\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4145491803278689\n", + "T-Statistic: -2.833861411958958\n", + "P-Value: 0.0659792994727473\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3423728813559322\n", + "T-Statistic: -2.203747318127119\n", + "P-Value: 0.15839077808830893\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1283\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1284\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.22916666666666669\n", + "T-Statistic: -8.901067221690772\n", + "P-Value: 0.0029905765083464166\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.2278954802259887\n", + "T-Statistic: -2.1351104338488116\n", + "P-Value: 0.12241769751046185\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1284\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1285\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.33749999999999997\n", + "T-Statistic: -2.4314657198035565\n", + "P-Value: 0.09320783628411884\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.28269774011299437\n", + "T-Statistic: -3.521966119583988\n", + "P-Value: 0.03886239529216442\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1285\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1286\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.27329234972677596\n", + "T-Statistic: -4.855988855019134\n", + "P-Value: 0.016672924204011234\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.19077212806026367\n", + "T-Statistic: -1.9859386750277492\n", + "P-Value: 0.1854304501080007\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1286\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1287\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.25341530054644806\n", + "T-Statistic: -13.539511357945873\n", + "P-Value: 0.0008713612072207959\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.21942090395480224\n", + "T-Statistic: -37.92307692307706\n", + "P-Value: 4.033425013157966e-05\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1287\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1288\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.3208333333333333\n", + "T-Statistic: -11.206006160936766\n", + "P-Value: 0.0015233714589831572\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.2909604519774011\n", + "T-Statistic: -2.6148419405355545\n", + "P-Value: 0.07935310501190229\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1288\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1289\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -3.5780996247983943\n", + "P-Value: 0.037336695763226686\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.31779661016949157\n", + "T-Statistic: -6.831300510639733\n", + "P-Value: 0.0064184892478205335\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1289\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1290\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.29583333333333334\n", + "T-Statistic: -13.315684846477943\n", + "P-Value: 0.0009154444688870522\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.23241525423728815\n", + "T-Statistic: -4.29193057069012\n", + "P-Value: 0.02325623974857282\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1290\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1291\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.3875\n", + "T-Statistic: -11.688828790772387\n", + "P-Value: 0.0013453380246460723\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3377118644067797\n", + "T-Statistic: -3.1733791192340908\n", + "P-Value: 0.0503496079687739\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1291\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1292\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.3316939890710382\n", + "T-Statistic: -5.9679306695524\n", + "P-Value: 0.009413595633304362\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.2364406779661017\n", + "T-Statistic: -3.377315745886016\n", + "P-Value: 0.04317682241293437\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1292\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1293\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -3.3534274040500196\n", + "P-Value: 0.04394693731715465\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.3247175141242937\n", + "T-Statistic: -5.643467443294131\n", + "P-Value: 0.011010270079289456\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1293\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1294\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.3896857923497268\n", + "T-Statistic: -3.9226085660313683\n", + "P-Value: 0.029474240628051502\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.34611581920903955\n", + "T-Statistic: -7.873610453398626\n", + "P-Value: 0.004268617658520372\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1294\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1295\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.3708333333333333\n", + "T-Statistic: -2.9809200108781417\n", + "P-Value: 0.058553923379340976\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3162429378531073\n", + "T-Statistic: -2.825094808781018\n", + "P-Value: 0.06645811724840166\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1295\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1296\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3403005464480874\n", + "T-Statistic: -2.952162851632012\n", + "P-Value: 0.05992028137117902\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.26617231638418076\n", + "T-Statistic: -2.337868480725624\n", + "P-Value: 0.10141825639016255\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1296\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1297\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.2653688524590164\n", + "T-Statistic: -16.038628368958843\n", + "P-Value: 0.0005271380369548524\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.18975988700564972\n", + "T-Statistic: -3.2917415848871014\n", + "P-Value: 0.04601775946765137\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1297\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1298\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4262295081967213\n", + "Average of Other Ratios: 0.3083333333333333\n", + "T-Statistic: -3.8269224330393166\n", + "P-Value: 0.03142577924163457\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.25741525423728817\n", + "T-Statistic: -3.8324745624304253\n", + "P-Value: 0.03130808733647537\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1298\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1299\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4144125683060109\n", + "T-Statistic: -3.4211013109598394\n", + "P-Value: 0.041809228433188725\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.576271186440678\n", + "Average of Other Ratios: 0.32902542372881355\n", + "T-Statistic: -9.542627417316496\n", + "P-Value: 0.0024409490317057526\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1299\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1300\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.32916666666666666\n", + "T-Statistic: -4.849227812278989\n", + "P-Value: 0.016736391339350645\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.27860169491525427\n", + "T-Statistic: -2.340709452706567\n", + "P-Value: 0.10115640234950059\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1300\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1301\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3152322404371585\n", + "T-Statistic: -17.69183502777838\n", + "P-Value: 0.00039371274155216776\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.27005649717514124\n", + "T-Statistic: -7.413571269033491\n", + "P-Value: 0.005077486364715219\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1301\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1302\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -11.68659984883048\n", + "P-Value: 0.001346094689122379\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.30798022598870056\n", + "T-Statistic: -3.889154254622231\n", + "P-Value: 0.030138373467386664\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1302\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1303\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -2.873384998124319\n", + "P-Value: 0.06387326471495713\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.30254237288135594\n", + "T-Statistic: -27.6666666666665\n", + "P-Value: 0.023000340589345674\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1303\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1304\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.4063524590163934\n", + "T-Statistic: -1.9862461743452617\n", + "P-Value: 0.14119767735260078\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.27464689265536724\n", + "T-Statistic: -3.0029751452311446\n", + "P-Value: 0.05753239842963967\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1304\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1305\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.31898907103825136\n", + "T-Statistic: -5.827473612902438\n", + "P-Value: 0.01006479830694792\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.24463276836158193\n", + "T-Statistic: -2.631494801492514\n", + "P-Value: 0.07822464952325936\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1305\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1306\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3939207650273224\n", + "T-Statistic: -3.0276117316398676\n", + "P-Value: 0.05641761781506673\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3432203389830508\n", + "T-Statistic: -2.8168388126916613\n", + "P-Value: 0.06691301311594693\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1306\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1307\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3358606557377049\n", + "T-Statistic: -4.734425236222974\n", + "P-Value: 0.017863689846178848\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.31779661016949157\n", + "T-Statistic: -5.039254603264428\n", + "P-Value: 0.015065771599115848\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1307\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1308\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3656420765027322\n", + "T-Statistic: -5.128769703376435\n", + "P-Value: 0.014353952607178531\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.2701271186440678\n", + "T-Statistic: -3.9729784680823\n", + "P-Value: 0.028509358260687255\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1308\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1309\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3566939890710382\n", + "T-Statistic: -7.883059676687519\n", + "P-Value: 0.004253852045425947\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.2951977401129944\n", + "T-Statistic: -6.340155233829669\n", + "P-Value: 0.00793564788180792\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1309\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1310\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -9.434713700912411\n", + "P-Value: 0.0025234427155974874\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.2615819209039548\n", + "T-Statistic: -3.256754496379134\n", + "P-Value: 0.04724737989320142\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1310\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1311\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.3041666666666667\n", + "T-Statistic: -10.096584056706222\n", + "P-Value: 0.0020692799372390666\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.2699152542372881\n", + "T-Statistic: -5.1240286090443545\n", + "P-Value: 0.014390539846078056\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1311\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1312\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.43333333333333335\n", + "T-Statistic: -3.508196721311475\n", + "P-Value: 0.0392487316994922\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.37563559322033896\n", + "T-Statistic: -2.224345699469886\n", + "P-Value: 0.11258691448891019\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1312\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1313\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.35\n", + "T-Statistic: -4.813749731486455\n", + "P-Value: 0.01707463310186129\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.29783427495291903\n", + "T-Statistic: -3.5957489244818297\n", + "P-Value: 0.06938932086375887\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1313\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1314\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.29166666666666663\n", + "T-Statistic: -7.216870606867377\n", + "P-Value: 0.005485187056437409\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.18568738229755177\n", + "T-Statistic: -1.9376663567761192\n", + "P-Value: 0.19225685599558717\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1314\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1315\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.3375\n", + "T-Statistic: -6.518811541138283\n", + "P-Value: 0.007334069087384167\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3333333333333333\n", + "Average of Other Ratios: 0.2796610169491526\n", + "T-Statistic: -3.6565517048676255\n", + "P-Value: 0.035330102302749224\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1315\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1316\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.22499999999999998\n", + "T-Statistic: -7.68678962084268\n", + "P-Value: 0.004574973719624272\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.26666666666666666\n", + "Average of Other Ratios: 0.1906779661016949\n", + "T-Statistic: -2.7138510050026126\n", + "P-Value: 0.07292684266802282\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1316\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1317\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.39166666666666666\n", + "T-Statistic: -9.409277800552035\n", + "P-Value: 0.0025434230249447316\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.37570621468926557\n", + "T-Statistic: -2.8482596056990577\n", + "P-Value: 0.06520219060265608\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1317\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1318\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.3541666666666667\n", + "T-Statistic: -21.360184505383586\n", + "P-Value: 0.00022451203065569885\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.29103107344632767\n", + "T-Statistic: -4.887164066279016\n", + "P-Value: 0.016384294164034276\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1318\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1319\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.2125\n", + "T-Statistic: -6.081034149138327\n", + "P-Value: 0.008928849971299894\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23333333333333334\n", + "Average of Other Ratios: 0.1483050847457627\n", + "T-Statistic: -2.866666666666667\n", + "P-Value: 0.06422527059785652\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1319\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1320\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3603142076502732\n", + "T-Statistic: -2.9539865796064393\n", + "P-Value: 0.05983245019134663\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.29209039548022603\n", + "T-Statistic: -7.654614504282301\n", + "P-Value: 0.016642001057945596\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1320\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1321\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.33333333333333337\n", + "T-Statistic: -8.952610373293831\n", + "P-Value: 0.0029407013393304916\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.2913135593220339\n", + "T-Statistic: -4.79001040774592\n", + "P-Value: 0.017305937126765883\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1321\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1322\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.39849726775956285\n", + "T-Statistic: -6.830325200725934\n", + "P-Value: 0.00642110575024065\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.28700564971751413\n", + "T-Statistic: -6.922217740406311\n", + "P-Value: 0.006180622722235997\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1322\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1323\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -3.737704918032786\n", + "P-Value: 0.03339722912081489\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3292372881355933\n", + "T-Statistic: -6.337740226443206\n", + "P-Value: 0.007944216283270866\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1323\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1324\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.2791666666666667\n", + "T-Statistic: -11.423030339825978\n", + "P-Value: 0.00143970284382488\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.2192090395480226\n", + "T-Statistic: -2.472975320624164\n", + "P-Value: 0.08982713951273004\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1324\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1325\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.39419398907103825\n", + "T-Statistic: -4.965110212591529\n", + "P-Value: 0.01569056902102536\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.33319209039548026\n", + "T-Statistic: -6.046351066103405\n", + "P-Value: 0.00907394218732086\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1325\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1326\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3604508196721311\n", + "T-Statistic: -3.3681723780241897\n", + "P-Value: 0.043469547365275946\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.269774011299435\n", + "T-Statistic: -2.6462479326196116\n", + "P-Value: 0.0772414424024133\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1326\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1327\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -3.3374687113774946\n", + "P-Value: 0.04447114763467144\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.30798022598870056\n", + "T-Statistic: -6.24495382880618\n", + "P-Value: 0.008282834737265843\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1327\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1328\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.34002732240437156\n", + "T-Statistic: -9.8986749455488\n", + "P-Value: 0.0021928507964472117\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.29936440677966103\n", + "T-Statistic: -4.542638876598001\n", + "P-Value: 0.019977054761155088\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1328\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1329\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -3.473480352013456\n", + "P-Value: 0.040244742555193926\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3305084745762712\n", + "T-Statistic: -3.092031441659996\n", + "P-Value: 0.05362839188122225\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1329\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1330\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.33189890710382514\n", + "T-Statistic: -10.490828508285672\n", + "P-Value: 0.001849332165580751\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.2662429378531074\n", + "T-Statistic: -2.0881715296001104\n", + "P-Value: 0.12800003155007977\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1330\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1331\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.3065573770491803\n", + "T-Statistic: -8.898873673993194\n", + "P-Value: 0.002992723821402328\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.29936440677966103\n", + "T-Statistic: -2.5346106753467175\n", + "P-Value: 0.08507981646292648\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1331\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1332\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.2708333333333333\n", + "T-Statistic: -11.732895329579321\n", + "P-Value: 0.0013304938857159276\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.2617231638418079\n", + "T-Statistic: -4.608368949689674\n", + "P-Value: 0.019217889743118922\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1332\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1333\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -10.562670498616757\n", + "P-Value: 0.0018126371846850279\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3415960451977401\n", + "T-Statistic: -3.626931567328918\n", + "P-Value: 0.036071124625796494\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1333\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1334\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.4628415300546448\n", + "T-Statistic: -5.357142857142865\n", + "P-Value: 0.11748359353956558\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.38022598870056495\n", + "T-Statistic: -2.535033549783168\n", + "P-Value: 0.08504832368725805\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1334\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1335\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.37336065573770494\n", + "T-Statistic: -4.6106699178095605\n", + "P-Value: 0.019191997918002076\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3209039548022599\n", + "T-Statistic: -3.4248301792021634\n", + "P-Value: 0.04169532187000911\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1335\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1336\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3980191256830601\n", + "T-Statistic: -2.348482665780014\n", + "P-Value: 0.10044412446307645\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3545197740112994\n", + "T-Statistic: -4.1111111111111125\n", + "P-Value: 0.02606395678518873\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1336\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1337\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -5.4804035870677845\n", + "P-Value: 0.011947625433148852\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.32457627118644067\n", + "T-Statistic: -6.742138664692472\n", + "P-Value: 0.006663608853032706\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1337\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1338\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.3333333333333333\n", + "T-Statistic: -3.1843068128816943\n", + "P-Value: 0.04992864292949258\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.288135593220339\n", + "T-Statistic: -2.6750562472452653\n", + "P-Value: 0.07536520355046247\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1338\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1339\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -4.59932645244697\n", + "P-Value: 0.01932008137256461\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3627118644067797\n", + "T-Statistic: -3.6442193578369366\n", + "P-Value: 0.03563624031042606\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1339\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1340\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -10.845912217186935\n", + "P-Value: 0.0016770220983140147\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.28241525423728814\n", + "T-Statistic: -3.0409065774469273\n", + "P-Value: 0.0558273074325418\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1340\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1341\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.2833333333333333\n", + "T-Statistic: -5.896420481369791\n", + "P-Value: 0.009738001288109581\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.2319915254237288\n", + "T-Statistic: -6.013276450562215\n", + "P-Value: 0.009215200973280897\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1341\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1342\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3812158469945356\n", + "T-Statistic: -3.7975738512734774\n", + "P-Value: 0.032057418473780255\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.34576271186440677\n", + "T-Statistic: -2.4308053376742573\n", + "P-Value: 0.09326286520252099\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1342\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1343\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.4291666666666667\n", + "T-Statistic: -30.452174863987732\n", + "P-Value: 7.779154477118327e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.40091807909604515\n", + "T-Statistic: -2.985011716688378\n", + "P-Value: 0.05836269929186945\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1343\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1344\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.22499999999999998\n", + "T-Statistic: -23.027757458005887\n", + "P-Value: 0.00017938051259614428\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.2192090395480226\n", + "T-Statistic: -4.357142857142859\n", + "P-Value: 0.02234002693058057\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1344\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1345\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.29583333333333334\n", + "T-Statistic: -3.202682563338301\n", + "P-Value: 0.04923073087668093\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.24456214689265537\n", + "T-Statistic: -2.609888745022117\n", + "P-Value: 0.07969262184669647\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1345\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1346\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.4817622950819672\n", + "T-Statistic: -4.70814887470562\n", + "P-Value: 0.018135489628524907\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.47238700564971753\n", + "T-Statistic: -1.8841000222390059\n", + "P-Value: 0.15606129783472963\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1346\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1347\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.3583333333333333\n", + "T-Statistic: -9.722669668020147\n", + "P-Value: 0.0023110927865077316\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3461864406779661\n", + "T-Statistic: -5.059912204313382\n", + "P-Value: 0.014897489104880094\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1347\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1348\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.32083333333333336\n", + "T-Statistic: -4.437542626227876\n", + "P-Value: 0.021273677300139238\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3093220338983051\n", + "T-Statistic: -3.3757575757575755\n", + "P-Value: 0.04322652987968803\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1348\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1349\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.4314890710382514\n", + "T-Statistic: -9.676632078012846\n", + "P-Value: 0.0023434092465502462\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3797316384180791\n", + "T-Statistic: -2.9703710359097775\n", + "P-Value: 0.05905056214606797\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1349\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1350\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.3208333333333333\n", + "T-Statistic: -3.604111013983922\n", + "P-Value: 0.03665562896226882\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.2362523540489642\n", + "T-Statistic: -3.955281402843921\n", + "P-Value: 0.05837978400303097\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1350\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1351\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.3208333333333333\n", + "T-Statistic: -21.583606557377053\n", + "P-Value: 0.00021764701504013405\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.29764595103578156\n", + "T-Statistic: -4.346534653465347\n", + "P-Value: 0.04906822409381993\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1351\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1352\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.41666666666666663\n", + "T-Statistic: -3.2125451549484705\n", + "P-Value: 0.04886123097830394\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.2911723163841808\n", + "T-Statistic: -4.423761578539075\n", + "P-Value: 0.021451702572010452\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1352\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1353\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.3208333333333333\n", + "T-Statistic: -8.591077485213033\n", + "P-Value: 0.0033154323787321117\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.27838983050847455\n", + "T-Statistic: -3.7753252988406856\n", + "P-Value: 0.03254713415412944\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1353\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1354\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.38545081967213113\n", + "T-Statistic: -2.9267563949002504\n", + "P-Value: 0.06116082615925972\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3586158192090395\n", + "T-Statistic: -2.541573388991179\n", + "P-Value: 0.08456310051618006\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1354\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1355\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3236338797814207\n", + "T-Statistic: -13.636558407121642\n", + "P-Value: 0.000853123683257044\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.2318502824858757\n", + "T-Statistic: -2.7486274334487737\n", + "P-Value: 0.07082370041149343\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1355\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1356\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.36516393442622946\n", + "T-Statistic: -8.46907603191196\n", + "P-Value: 0.0034560758139621692\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.29936440677966103\n", + "T-Statistic: -2.1631195885543937\n", + "P-Value: 0.11922505386124203\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1356\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1357\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.36065573770491804\n", + "T-Statistic: -3.1934829897182784\n", + "P-Value: 0.04957857968168316\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.30508474576271183\n", + "T-Statistic: -12.091525958289973\n", + "P-Value: 0.0012174072403681394\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1357\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1358\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3235655737704918\n", + "T-Statistic: -28.52854660180471\n", + "P-Value: 9.456145255444976e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.29103107344632767\n", + "T-Statistic: -3.7674177162295606\n", + "P-Value: 0.03272349692653532\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1358\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1359\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.4291666666666667\n", + "T-Statistic: -2.0643127364438847\n", + "P-Value: 0.13095346493752535\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.35861581920903957\n", + "T-Statistic: -3.110819906677478\n", + "P-Value: 0.05284775046942044\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1359\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1360\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.38148907103825136\n", + "T-Statistic: -1.9632892320372335\n", + "P-Value: 0.1443881711602152\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.2809792843691149\n", + "T-Statistic: -5.736842105263147\n", + "P-Value: 0.02906640150529952\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1360\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1361\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.3028005464480874\n", + "T-Statistic: -21.112622602464683\n", + "P-Value: 0.00023245973293610858\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.2526365348399247\n", + "T-Statistic: -3.4171779141104293\n", + "P-Value: 0.07600312612521545\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1361\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1362\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -3.54807654763531\n", + "P-Value: 0.038143015996558144\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3220338983050848\n", + "T-Statistic: -3.53121005385469\n", + "P-Value: 0.03860574928821704\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1362\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1363\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.38155737704918036\n", + "T-Statistic: -4.0869440858831725\n", + "P-Value: 0.026471716786619498\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.314406779661017\n", + "T-Statistic: -1.800059978008079\n", + "P-Value: 0.2136563487360759\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1363\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1364\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3941256830601093\n", + "T-Statistic: -5.534321299197799\n", + "P-Value: 0.011626515414883042\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3840395480225989\n", + "T-Statistic: -2.2968348396222584\n", + "P-Value: 0.10529360084626542\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1364\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1365\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.29583333333333334\n", + "T-Statistic: -9.689747679740668\n", + "P-Value: 0.002334141978546938\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.2153954802259887\n", + "T-Statistic: -4.628137283451674\n", + "P-Value: 0.01899691387142354\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1365\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1366\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.32083333333333336\n", + "T-Statistic: -2.9505777976622465\n", + "P-Value: 0.059996749206154006\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.2701271186440678\n", + "T-Statistic: -2.5462474269165756\n", + "P-Value: 0.08421839435807502\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1366\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1367\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.3375\n", + "T-Statistic: -3.780327868852458\n", + "P-Value: 0.03243619070712851\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.288135593220339\n", + "T-Statistic: -3.421943449225448\n", + "P-Value: 0.04178346889141796\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1367\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1368\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -3.8488075064770464\n", + "P-Value: 0.03096513128016824\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.29992937853107343\n", + "T-Statistic: -1.9390607807005154\n", + "P-Value: 0.1478476458083794\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1368\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1369\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -4.878136536726772\n", + "P-Value: 0.01646720054964036\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.26935028248587567\n", + "T-Statistic: -1.643431635388741\n", + "P-Value: 0.34799773853532184\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1369\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1370\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3066256830601093\n", + "T-Statistic: -3.3297523730648932\n", + "P-Value: 0.044727459212230945\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.22761299435028248\n", + "T-Statistic: -4.4183200488016485\n", + "P-Value: 0.021522528086765566\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1370\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1371\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.35252732240437157\n", + "T-Statistic: -2.5701626159312543\n", + "P-Value: 0.08248147008467596\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.2528248587570622\n", + "T-Statistic: -5.471809142918229\n", + "P-Value: 0.011999877727690598\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1371\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1372\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.36666666666666664\n", + "T-Statistic: -5.403746742275941\n", + "P-Value: 0.012424423736674949\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3093220338983051\n", + "T-Statistic: -13.229901852633054\n", + "P-Value: 0.0009331248455100845\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1372\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1373\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.41857923497267757\n", + "T-Statistic: -2.837173206379461\n", + "P-Value: 0.06579953452896789\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3474576271186441\n", + "T-Statistic: -4.715027198381945\n", + "P-Value: 0.018063824737018258\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1373\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1374\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.2981785063752277\n", + "T-Statistic: -9.104294478527612\n", + "P-Value: 0.011850419592490571\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.2699858757062147\n", + "T-Statistic: -5.869851287372005\n", + "P-Value: 0.009862263796097724\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1374\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1375\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3736338797814208\n", + "T-Statistic: -3.6168425013226613\n", + "P-Value: 0.03632805703920555\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.2923728813559322\n", + "T-Statistic: -5.133234383709284\n", + "P-Value: 0.014319609315535732\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1375\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1376\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.3458333333333333\n", + "T-Statistic: -9.297077569799361\n", + "P-Value: 0.0026341027851884773\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3040960451977401\n", + "T-Statistic: -2.8752211601887643\n", + "P-Value: 0.06377747883646023\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1376\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1377\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -3.0118665582406012\n", + "P-Value: 0.05712690752613409\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3248587570621469\n", + "T-Statistic: -1.9896995023342197\n", + "P-Value: 0.14072495503830776\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1377\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1378\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3567622950819672\n", + "T-Statistic: -2.922322679137377\n", + "P-Value: 0.061380599938098745\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.2953389830508475\n", + "T-Statistic: -4.1824309711049805\n", + "P-Value: 0.024906888121385386\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1378\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1379\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.3567622950819672\n", + "T-Statistic: -8.560154259731128\n", + "P-Value: 0.003350353702966942\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.27429378531073445\n", + "T-Statistic: -4.389736277617274\n", + "P-Value: 0.021899581152473327\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1379\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1380\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4426229508196721\n", + "Average of Other Ratios: 0.3958333333333333\n", + "T-Statistic: -2.1349234997177082\n", + "P-Value: 0.12243934625158044\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.30360169491525424\n", + "T-Statistic: -6.426920856371511\n", + "P-Value: 0.007635697442918866\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1380\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1381\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3274590163934426\n", + "T-Statistic: -7.792967812252545\n", + "P-Value: 0.004397428835019784\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.2955508474576271\n", + "T-Statistic: -3.519233773599044\n", + "P-Value: 0.03893867159865646\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1381\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1382\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4262295081967213\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -2.3199911010934016\n", + "P-Value: 0.10308502717775488\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3082627118644068\n", + "T-Statistic: -4.022350177849458\n", + "P-Value: 0.02760267623485285\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1382\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1383\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.38155737704918036\n", + "T-Statistic: -4.19677462214937\n", + "P-Value: 0.02468223424934266\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.34821092278719395\n", + "T-Statistic: -4.510204081632659\n", + "P-Value: 0.045808057902316804\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1383\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1384\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -9.637017116211096\n", + "P-Value: 0.002371698207192692\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3258003766478343\n", + "T-Statistic: -4.913538149119954\n", + "P-Value: 0.039012347759055314\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1384\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1385\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.28724954462659374\n", + "T-Statistic: -20.478625837697965\n", + "P-Value: 0.0023760107227863043\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.24887005649717514\n", + "T-Statistic: -3.297939899442541\n", + "P-Value: 0.04580416394678644\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1385\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1386\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.39419398907103825\n", + "T-Statistic: -23.001886718608972\n", + "P-Value: 0.00017998371795611563\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.29209039548022603\n", + "T-Statistic: -9.26047768699887\n", + "P-Value: 0.011460849730983888\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1386\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1387\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.32916666666666666\n", + "T-Statistic: -6.6941484555943065\n", + "P-Value: 0.006800654936853843\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3250706214689265\n", + "T-Statistic: -4.167794334654786\n", + "P-Value: 0.025138858812902672\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1387\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1388\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -10.355237394824906\n", + "P-Value: 0.001921317535318784\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3121468926553672\n", + "T-Statistic: -3.4545091251635194\n", + "P-Value: 0.04080264000619044\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1388\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1389\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4269808743169399\n", + "T-Statistic: -3.4667216700978054\n", + "P-Value: 0.04044237885506118\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.385593220338983\n", + "T-Statistic: -5.066666666666667\n", + "P-Value: 0.014842997288171941\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1389\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1390\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3236338797814208\n", + "T-Statistic: -3.3358653417919237\n", + "P-Value: 0.04452425266445473\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.2530367231638418\n", + "T-Statistic: -3.0699274770848146\n", + "P-Value: 0.05456547495842803\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1390\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1391\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.28565573770491803\n", + "T-Statistic: -2.3900038580000214\n", + "P-Value: 0.0967410413896404\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.2402542372881356\n", + "T-Statistic: -2.253658340717321\n", + "P-Value: 0.10956651809067314\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1391\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1392\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.3323087431693989\n", + "T-Statistic: -10.546963974844278\n", + "P-Value: 0.001820576611444715\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3220338983050848\n", + "T-Statistic: -2.732807218615621\n", + "P-Value: 0.07177097972724268\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1392\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1393\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.39166666666666666\n", + "T-Statistic: -12.616523095570006\n", + "P-Value: 0.0010737810472414555\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.32069209039548024\n", + "T-Statistic: -7.1384266143873925\n", + "P-Value: 0.005659809403554773\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1393\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1394\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.331511839708561\n", + "T-Statistic: -2.9439949370436143\n", + "P-Value: 0.09860789571567317\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.24152542372881355\n", + "T-Statistic: -5.9603956067927\n", + "P-Value: 0.027012812355076195\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1394\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1395\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.819672131147541\n", + "Average of Other Ratios: 0.325\n", + "T-Statistic: -13.385460331466769\n", + "P-Value: 0.0009013902444265222\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.29145480225988696\n", + "T-Statistic: -3.4168337494506282\n", + "P-Value: 0.04194007659421182\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1395\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1396\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.19508196721311474\n", + "T-Statistic: -6.738703907451196\n", + "P-Value: 0.00667329610025673\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.1864406779661017\n", + "Average of Other Ratios: 0.13495762711864406\n", + "T-Statistic: -2.257918405893237\n", + "P-Value: 0.10913572969388878\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1396\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1397\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.31666666666666665\n", + "T-Statistic: -9.776873289560362\n", + "P-Value: 0.0022737966110181968\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3050847457627119\n", + "Average of Other Ratios: 0.23644067796610171\n", + "T-Statistic: -3.233110040049279\n", + "P-Value: 0.04810198076292131\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1397\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1398\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.2782103825136612\n", + "T-Statistic: -6.842713959859326\n", + "P-Value: 0.006387974012842435\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.27429378531073445\n", + "T-Statistic: -2.907350507765195\n", + "P-Value: 0.062130092881679595\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1398\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1399\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.31058743169398906\n", + "T-Statistic: -7.0022122373829845\n", + "P-Value: 0.005980846181537664\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.24053672316384178\n", + "T-Statistic: -2.365917408343781\n", + "P-Value: 0.0988685692030821\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1399\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1400\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.2774590163934426\n", + "T-Statistic: -7.817198672088378\n", + "P-Value: 0.004358191490211607\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.24915254237288134\n", + "T-Statistic: -3.4558397655933613\n", + "P-Value: 0.04076318933119888\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1400\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1401\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -3.7510920776385794\n", + "P-Value: 0.033091499306512666\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.34611581920903955\n", + "T-Statistic: -2.894903284380277\n", + "P-Value: 0.06276192114601883\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1401\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1402\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.41666666666666663\n", + "T-Statistic: -5.0158308909899345\n", + "P-Value: 0.015259597669989054\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.34201977401129946\n", + "T-Statistic: -2.394619302503736\n", + "P-Value: 0.09633975086570609\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1402\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1403\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3396174863387978\n", + "T-Statistic: -3.020014584938689\n", + "P-Value: 0.05675846509350086\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.27853107344632766\n", + "T-Statistic: -5.181815900134804\n", + "P-Value: 0.013952747029258906\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1403\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1404\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.35416666666666663\n", + "T-Statistic: -6.420806830530328\n", + "P-Value: 0.0076563407117634\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3686440677966102\n", + "T-Statistic: -4.46962233410428\n", + "P-Value: 0.020866618588713408\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1404\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1405\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.42704918032786887\n", + "T-Statistic: -3.2041931426214503\n", + "P-Value: 0.04917390886551068\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3754943502824859\n", + "T-Statistic: -3.4249037287283453\n", + "P-Value: 0.041693079107582236\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1405\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1406\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.38995901639344266\n", + "T-Statistic: -3.129964263210284\n", + "P-Value: 0.0520669925513877\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3121468926553672\n", + "T-Statistic: -9.141660034508423\n", + "P-Value: 0.00276691986404339\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1406\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1407\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.44002732240437165\n", + "T-Statistic: -7.029706101430858\n", + "P-Value: 0.005914150507692506\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3968926553672316\n", + "T-Statistic: -2.528902694294367\n", + "P-Value: 0.08550631275341908\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1407\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1408\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3360655737704918\n", + "T-Statistic: -6.662521026761018\n", + "P-Value: 0.006893006911496739\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.25\n", + "T-Statistic: -14.066533632030769\n", + "P-Value: 0.0007781509499729613\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1408\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1409\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.41120218579234974\n", + "T-Statistic: -4.744558099428093\n", + "P-Value: 0.01776029146137704\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3584039548022599\n", + "T-Statistic: -3.1685329501250488\n", + "P-Value: 0.050537730061605154\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1409\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1410\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.3916666666666667\n", + "T-Statistic: -11.23243861284888\n", + "P-Value: 0.0015128416306198192\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.38848870056497176\n", + "T-Statistic: -3.0811823304553716\n", + "P-Value: 0.05408577968425568\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1410\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1411\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.4360655737704918\n", + "T-Statistic: -4.896859345148878\n", + "P-Value: 0.01629585933682827\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.40529661016949153\n", + "T-Statistic: -3.1510846069279967\n", + "P-Value: 0.05122244392981766\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1411\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1412\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.41065573770491803\n", + "T-Statistic: -3.340359673429475\n", + "P-Value: 0.04437559930810233\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.2995762711864407\n", + "T-Statistic: -6.934810075795833\n", + "P-Value: 0.006148597978661057\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1412\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1413\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -3.970491803278687\n", + "P-Value: 0.02855603110389197\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.34180790960451973\n", + "T-Statistic: -6.35085296108589\n", + "P-Value: 0.007897838244939661\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1413\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1414\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -4.954036545154963\n", + "P-Value: 0.015786763366299392\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.34209039548022596\n", + "T-Statistic: -5.7423119859693115\n", + "P-Value: 0.010488586403531558\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1414\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1415\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.439275956284153\n", + "T-Statistic: -2.5509371831068104\n", + "P-Value: 0.0838742637314232\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3686440677966102\n", + "T-Statistic: -6.342784542369049\n", + "P-Value: 0.007926332946572029\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1415\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1416\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -12.587812138042343\n", + "P-Value: 0.001081035521070716\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3\n", + "T-Statistic: -4.012310343727843\n", + "P-Value: 0.027784012310922687\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1416\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1417\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.42083333333333334\n", + "T-Statistic: -13.599864280438059\n", + "P-Value: 0.000859959220110807\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.4219632768361582\n", + "T-Statistic: -9.741023075752745\n", + "P-Value: 0.0022983741002138604\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1417\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1418\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.49330601092896176\n", + "T-Statistic: -2.4421197772396783\n", + "P-Value: 0.09232550216594695\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4518361581920904\n", + "T-Statistic: -2.3115409557535824\n", + "P-Value: 0.10388441475596431\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1418\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1419\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3809653916211293\n", + "T-Statistic: -1.5855147752916146\n", + "P-Value: 0.25372930058134713\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.31257062146892656\n", + "T-Statistic: -2.208880071004421\n", + "P-Value: 0.11422087778664086\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1419\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1420\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.32083333333333336\n", + "T-Statistic: -18.273075151598423\n", + "P-Value: 0.0003575799805904938\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3671610169491525\n", + "T-Statistic: -3.9629629629629703\n", + "P-Value: 0.02869794241127541\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1420\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1421\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4062841530054645\n", + "T-Statistic: -6.551255287390875\n", + "P-Value: 0.007231321491454494\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3545197740112994\n", + "T-Statistic: -4.900769721140663\n", + "P-Value: 0.01626036695364968\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1421\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1422\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.425\n", + "T-Statistic: -3.0865517657726187\n", + "P-Value: 0.05385879831750464\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.4194915254237288\n", + "T-Statistic: -2.0960998088359606\n", + "P-Value: 0.12703618205803208\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1422\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1423\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.819672131147541\n", + "Average of Other Ratios: 0.3875\n", + "T-Statistic: -15.69614416660204\n", + "P-Value: 0.000562059046312007\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4010593220338983\n", + "T-Statistic: -4.205211154248234\n", + "P-Value: 0.024551320079546985\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1423\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1424\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.4394808743169399\n", + "T-Statistic: -6.100988988496524\n", + "P-Value: 0.008846749505079436\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.4175141242937853\n", + "T-Statistic: -2.3260164970888964\n", + "P-Value: 0.10251957254301898\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1424\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1425\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.4354508196721312\n", + "T-Statistic: -8.022565688606122\n", + "P-Value: 0.004043498406787921\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3879237288135593\n", + "T-Statistic: -2.9208912467506574\n", + "P-Value: 0.06145176525841497\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1425\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1426\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.45833333333333337\n", + "T-Statistic: -7.5244830164877445\n", + "P-Value: 0.004865119180322468\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.41101694915254233\n", + "T-Statistic: -2.231327750334267\n", + "P-Value: 0.11185847276008758\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1426\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1427\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3650273224043715\n", + "T-Statistic: -11.968876772692552\n", + "P-Value: 0.0012545933074113616\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.2995056497175141\n", + "T-Statistic: -2.210333134870083\n", + "P-Value: 0.11406615150921069\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1427\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1428\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.32916666666666666\n", + "T-Statistic: -6.2514540023238165\n", + "P-Value: 0.008258502009868794\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.29774011299435027\n", + "T-Statistic: -3.952777215734684\n", + "P-Value: 0.05844741686027941\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1428\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1429\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.38176229508196724\n", + "T-Statistic: -9.8479960675198\n", + "P-Value: 0.0022260589594432483\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3432203389830508\n", + "T-Statistic: -3.021617032571375\n", + "P-Value: 0.05668635618881769\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1429\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1430\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.40710382513661203\n", + "T-Statistic: -3.2566855663164747\n", + "P-Value: 0.047249843238845575\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3079802259887006\n", + "T-Statistic: -34.12195121951229\n", + "P-Value: 5.5338575696591484e-05\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1430\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1431\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.44781420765027324\n", + "T-Statistic: -2.777298311206574\n", + "P-Value: 0.0691462057873957\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.4110169491525424\n", + "T-Statistic: -6.858659644654502\n", + "P-Value: 0.006345660358547117\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1431\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1432\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.38749999999999996\n", + "T-Statistic: -4.973957462489661\n", + "P-Value: 0.015614263014698614\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.33799435028248587\n", + "T-Statistic: -1.8646963468092161\n", + "P-Value: 0.15908733357598287\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1432\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1433\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.4034608378870674\n", + "T-Statistic: -5.908250072050177\n", + "P-Value: 0.027472183908806763\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.39887005649717516\n", + "T-Statistic: -2.8461538461538454\n", + "P-Value: 0.1044600519927322\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1433\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1434\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.41933060109289616\n", + "T-Statistic: -6.341832402564622\n", + "P-Value: 0.007929704462389898\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.3629237288135593\n", + "T-Statistic: -10.808536082221455\n", + "P-Value: 0.0016941290870239134\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1434\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1435\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4229508196721311\n", + "T-Statistic: -4.809523809523813\n", + "P-Value: 0.017115512915415635\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3291666666666667\n", + "T-Statistic: -5.281280870229153\n", + "P-Value: 0.01323906907450259\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1435\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1436\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3479052823315118\n", + "T-Statistic: -5.862500000000004\n", + "P-Value: 0.02788479615419901\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.2950564971751412\n", + "T-Statistic: -2.5561253487432714\n", + "P-Value: 0.08349557275673589\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1436\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1437\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.44166666666666665\n", + "T-Statistic: -6.172915501073344\n", + "P-Value: 0.008558915834601189\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.37132768361581925\n", + "T-Statistic: -12.168997094322684\n", + "P-Value: 0.0011946701728581847\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1437\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1438\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.425\n", + "T-Statistic: -2.836410863828114\n", + "P-Value: 0.06584086048198345\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3586158192090395\n", + "T-Statistic: -3.8295559807933683\n", + "P-Value: 0.03136988366637992\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1438\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1439\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.40607923497267756\n", + "T-Statistic: -3.337367049030468\n", + "P-Value: 0.044474512395329985\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3669491525423728\n", + "T-Statistic: -5.570197507360285\n", + "P-Value: 0.01141908670501291\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1439\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1440\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.3125\n", + "T-Statistic: -11.156636603647863\n", + "P-Value: 0.0015433006224139084\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.30819209039548023\n", + "T-Statistic: -2.607416226140155\n", + "P-Value: 0.07986277130041312\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1440\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1441\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.39999999999999997\n", + "T-Statistic: -2.3612943568800553\n", + "P-Value: 0.09928340428129473\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3813559322033898\n", + "T-Statistic: -4.548172491877701\n", + "P-Value: 0.019911659585532055\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1441\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1442\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3940573770491803\n", + "T-Statistic: -7.029354227468431\n", + "P-Value: 0.005914997890496912\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.29124293785310734\n", + "T-Statistic: -3.602392726796967\n", + "P-Value: 0.036700127748095945\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1442\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1443\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4853142076502732\n", + "T-Statistic: -3.218376091512211\n", + "P-Value: 0.04864442449363043\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3882062146892656\n", + "T-Statistic: -7.414941042518773\n", + "P-Value: 0.00507478976632389\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1443\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1444\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3277322404371585\n", + "T-Statistic: -7.5308054776588165\n", + "P-Value: 0.004853369315234331\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.27888418079096045\n", + "T-Statistic: -2.9282749062301643\n", + "P-Value: 0.06108578193839565\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1444\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1445\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4401639344262295\n", + "T-Statistic: -2.2348801205435334\n", + "P-Value: 0.11149003001722149\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3502824858757062\n", + "T-Statistic: -6.487446070815471\n", + "P-Value: 0.007435234784132052\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1445\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1446\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.38333333333333336\n", + "T-Statistic: -9.35437505152399\n", + "P-Value: 0.0025872696994375557\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.4301553672316384\n", + "T-Statistic: -2.6869021956079524\n", + "P-Value: 0.07461008165667392\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1446\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1447\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.35273224043715845\n", + "T-Statistic: -2.771421867769551\n", + "P-Value: 0.06948596977369918\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3220338983050848\n", + "T-Statistic: -2.888948165920659\n", + "P-Value: 0.06306704716766874\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1447\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1448\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.3916666666666667\n", + "T-Statistic: -10.246669168835966\n", + "P-Value: 0.0019816576756549577\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3163841807909605\n", + "T-Statistic: -13.063945294843638\n", + "P-Value: 0.0009686387721898685\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1448\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1449\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3605191256830601\n", + "T-Statistic: -5.027382355038384\n", + "P-Value: 0.015163608282268311\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.2754237288135593\n", + "T-Statistic: -11.666666666666663\n", + "P-Value: 0.007266951354550622\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1449\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1450\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.4354508196721312\n", + "T-Statistic: -2.2311148149182007\n", + "P-Value: 0.11188060441585682\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.41779661016949154\n", + "T-Statistic: -2.9499240992842286\n", + "P-Value: 0.060028321215766844\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1450\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1451\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.385655737704918\n", + "T-Statistic: -7.0347669547746365\n", + "P-Value: 0.005901980635791814\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.364406779661017\n", + "T-Statistic: -2.193378465041791\n", + "P-Value: 0.1158873271135981\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1451\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1452\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.38749999999999996\n", + "T-Statistic: -2.929866094565433\n", + "P-Value: 0.06100726967849578\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.364406779661017\n", + "T-Statistic: -3.2485009805585183\n", + "P-Value: 0.04754349406047413\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1452\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1453\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.35\n", + "T-Statistic: -20.62605763263766\n", + "P-Value: 0.0002492064346505477\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3038841807909604\n", + "T-Statistic: -7.310266791966458\n", + "P-Value: 0.0052864231861055435\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1453\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1454\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -3.46728445878972\n", + "P-Value: 0.040425874782353266\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.303954802259887\n", + "T-Statistic: -5.811865258054232\n", + "P-Value: 0.010140767780751686\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1454\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1455\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.3833333333333333\n", + "T-Statistic: -5.862266729790863\n", + "P-Value: 0.00989811735639771\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3931261770244821\n", + "T-Statistic: -2.400829779247631\n", + "P-Value: 0.1383734677102449\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1455\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1456\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.3875\n", + "T-Statistic: -4.478018111973132\n", + "P-Value: 0.02076175387541808\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3542372881355932\n", + "T-Statistic: -2.708482575649219\n", + "P-Value: 0.0732583731798295\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1456\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1457\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.40225409836065573\n", + "T-Statistic: -3.1045263606689577\n", + "P-Value: 0.05310763670073003\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3334039548022599\n", + "T-Statistic: -3.7816368562219433\n", + "P-Value: 0.0324072409590771\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1457\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1458\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.39371584699453555\n", + "T-Statistic: -2.5092989392309506\n", + "P-Value: 0.08699122587945884\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3794491525423729\n", + "T-Statistic: -2.2802263709416946\n", + "P-Value: 0.10691298923861156\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1458\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1459\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.3458333333333333\n", + "T-Statistic: -7.3958473467775\n", + "P-Value: 0.005112549560971972\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.35416666666666663\n", + "T-Statistic: -2.7464616338694436\n", + "P-Value: 0.07095246240968993\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1459\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1460\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.39999999999999997\n", + "T-Statistic: -2.7428341074780405\n", + "P-Value: 0.07116877964790087\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3728813559322034\n", + "T-Statistic: -2.118915741970083\n", + "P-Value: 0.1243102693621582\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1460\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1461\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -3.7454630727011047\n", + "P-Value: 0.03321961329360707\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.37153954802259886\n", + "T-Statistic: -2.386591892485171\n", + "P-Value: 0.09703900448133006\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1461\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1462\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3815573770491803\n", + "T-Statistic: -5.86945467255433\n", + "P-Value: 0.009864134442699606\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3502824858757062\n", + "T-Statistic: -8.101627221513192\n", + "P-Value: 0.003930361543241993\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1462\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1463\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.35\n", + "T-Statistic: -2.4617916243617888\n", + "P-Value: 0.09072301062363927\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.33742937853107347\n", + "T-Statistic: -3.6447947158122775\n", + "P-Value: 0.035621882387353185\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1463\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1464\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -4.8396027027399\n", + "P-Value: 0.016827287331982445\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.2951977401129944\n", + "T-Statistic: -9.61483104773844\n", + "P-Value: 0.002387739072267894\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1464\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1465\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.38155737704918036\n", + "T-Statistic: -4.355025558660401\n", + "P-Value: 0.02236903359688658\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.31207627118644066\n", + "T-Statistic: -4.941615370705469\n", + "P-Value: 0.015895579161855126\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1465\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1466\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.39999999999999997\n", + "T-Statistic: -3.694312398951678\n", + "P-Value: 0.03441340625590583\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.35903954802259885\n", + "T-Statistic: -3.4967484138936817\n", + "P-Value: 0.03957367708757962\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1466\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1467\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -8.04851009961339\n", + "P-Value: 0.004005903148525875\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.36864406779661013\n", + "T-Statistic: -5.216423434339635\n", + "P-Value: 0.013698860156045016\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1467\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1468\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.4166666666666667\n", + "T-Statistic: -3.9653624934399807\n", + "P-Value: 0.028652615314166956\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3813559322033898\n", + "T-Statistic: -10.623244953089115\n", + "P-Value: 0.0017824441583808043\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1468\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1469\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.3541666666666667\n", + "T-Statistic: -5.938681961594909\n", + "P-Value: 0.009544543156865669\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.2919020715630885\n", + "T-Statistic: -2.841519981956887\n", + "P-Value: 0.10474932255486497\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1469\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1470\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -3.9840670207259876\n", + "P-Value: 0.028302424799336666\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.33681732580037665\n", + "T-Statistic: -2.5438493495451917\n", + "P-Value: 0.12598335802475616\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1470\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1471\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.3528688524590164\n", + "T-Statistic: -12.365301190960757\n", + "P-Value: 0.001139527243227325\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3456920903954802\n", + "T-Statistic: -3.165877220586101\n", + "P-Value: 0.0506411985161682\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1471\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1472\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -7.995723638492874\n", + "P-Value: 0.004082886586878434\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.29774011299435027\n", + "T-Statistic: -5.144479879486528\n", + "P-Value: 0.03576976981268908\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1472\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1473\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.5458333333333334\n", + "T-Statistic: -3.518727928731812\n", + "P-Value: 0.03895281371803606\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4957627118644068\n", + "T-Statistic: -4.919999999999999\n", + "P-Value: 0.016087280685965905\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1473\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1474\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3894125683060109\n", + "T-Statistic: -4.405715742078374\n", + "P-Value: 0.021687749143792793\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.36292372881355933\n", + "T-Statistic: -2.9112467789048893\n", + "P-Value: 0.06193395217425993\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1474\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1475\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.36250000000000004\n", + "T-Statistic: -3.7715052757850085\n", + "P-Value: 0.032632179601749925\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.28721751412429375\n", + "T-Statistic: -4.6372424549345395\n", + "P-Value: 0.018896243004971457\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1475\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1476\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.360724043715847\n", + "T-Statistic: -4.619815242020446\n", + "P-Value: 0.01908953586981084\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3077683615819209\n", + "T-Statistic: -3.416406029443321\n", + "P-Value: 0.04195321965391951\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1476\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1477\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.41475409836065574\n", + "T-Statistic: -5.369818568286147\n", + "P-Value: 0.012643397039975352\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -6.968395313620221\n", + "P-Value: 0.006064246319147875\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1477\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1478\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.4059426229508197\n", + "T-Statistic: -4.851990369531891\n", + "P-Value: 0.016710420799139213\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.325\n", + "T-Statistic: -2.3275547317374796\n", + "P-Value: 0.10237581960629237\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1478\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1479\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4314207650273224\n", + "T-Statistic: -5.6998934006029405\n", + "P-Value: 0.01070840976220116\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.38201506591337103\n", + "T-Statistic: -2.838985414963086\n", + "P-Value: 0.10490802421935423\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1479\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1480\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.45642076502732243\n", + "T-Statistic: -8.972601513715789\n", + "P-Value: 0.0029216534818719115\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3797316384180791\n", + "T-Statistic: -6.404709068596953\n", + "P-Value: 0.007711044857755044\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1480\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1481\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.3208333333333333\n", + "T-Statistic: -14.29409263370358\n", + "P-Value: 0.0007419958228994995\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.2909604519774011\n", + "T-Statistic: -3.081578172139684\n", + "P-Value: 0.054069005258383575\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1481\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1482\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.4166666666666667\n", + "T-Statistic: -9.335911788157421\n", + "P-Value: 0.002602239684135932\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.40254237288135597\n", + "T-Statistic: -5.266283362867523\n", + "P-Value: 0.013343577598251742\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1482\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1483\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3610655737704918\n", + "T-Statistic: -3.939648677119553\n", + "P-Value: 0.029143186739241125\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3587570621468927\n", + "T-Statistic: -3.544745038970269\n", + "P-Value: 0.038233853222009985\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1483\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1484\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4394808743169399\n", + "T-Statistic: -2.7199087833206432\n", + "P-Value: 0.07255497251636857\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3248587570621469\n", + "T-Statistic: -2.2831482556870473\n", + "P-Value: 0.10662592307782301\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1484\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1485\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.32766393442622954\n", + "T-Statistic: -7.616943405099432\n", + "P-Value: 0.004696954329420175\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.2923728813559322\n", + "T-Statistic: -4.252457958970371\n", + "P-Value: 0.023834502002819766\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1485\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1486\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.41454918032786886\n", + "T-Statistic: -4.203244682830465\n", + "P-Value: 0.024581754588030094\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.40084745762711865\n", + "T-Statistic: -3.941176470588236\n", + "P-Value: 0.029113738898400077\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1486\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1487\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.36666666666666664\n", + "T-Statistic: -8.088416484282563\n", + "P-Value: 0.003948972398783288\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.32090395480225986\n", + "T-Statistic: -5.31174648930179\n", + "P-Value: 0.013030041853782981\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1487\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1488\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.35601092896174863\n", + "T-Statistic: -2.719036894075117\n", + "P-Value: 0.07260835021632396\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3204802259887006\n", + "T-Statistic: -2.1906690113295393\n", + "P-Value: 0.11618158740373133\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1488\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1489\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.3983606557377049\n", + "T-Statistic: -10.650424052370854\n", + "P-Value: 0.001769113584875861\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.325\n", + "T-Statistic: -3.9643008779307634\n", + "P-Value: 0.028672657850254017\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1489\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1490\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.43558743169398906\n", + "T-Statistic: -2.9888817184112546\n", + "P-Value: 0.058182556819309464\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3544491525423729\n", + "T-Statistic: -7.463121733660972\n", + "P-Value: 0.00498112898453179\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1490\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1491\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3937841530054645\n", + "T-Statistic: -4.000584919681885\n", + "P-Value: 0.027997738989114628\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3686440677966102\n", + "T-Statistic: -4.69122943243946\n", + "P-Value: 0.018313350909407743\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1491\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1492\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.3958333333333333\n", + "T-Statistic: -5.43407451169032\n", + "P-Value: 0.012232865280609554\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3626412429378531\n", + "T-Statistic: -2.5444250028895907\n", + "P-Value: 0.08435259068095222\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1492\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1493\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.5166666666666667\n", + "T-Statistic: -5.7882292798767665\n", + "P-Value: 0.010257246511227655\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.576271186440678\n", + "Average of Other Ratios: 0.4848870056497175\n", + "T-Statistic: -2.3523540362420694\n", + "P-Value: 0.10009165403065308\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1493\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1494\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3567622950819672\n", + "T-Statistic: -20.143694223280193\n", + "P-Value: 0.00026743218087057065\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.303954802259887\n", + "T-Statistic: -3.517707919348613\n", + "P-Value: 0.038981350469306286\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1494\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1495\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3688524590163934\n", + "T-Statistic: -2.2635535190344163\n", + "P-Value: 0.10856902810055359\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3038135593220339\n", + "T-Statistic: -4.198379191556001\n", + "P-Value: 0.024657266011800877\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1495\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1496\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.3791666666666667\n", + "T-Statistic: -5.261267526539542\n", + "P-Value: 0.013378770206243135\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.32923728813559316\n", + "T-Statistic: -5.019825255742889\n", + "P-Value: 0.015226316305783647\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1496\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1497\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4192622950819672\n", + "T-Statistic: -4.998024144164243\n", + "P-Value: 0.01540912351567112\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.33757062146892663\n", + "T-Statistic: -1.955570219094377\n", + "P-Value: 0.18968621213880818\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1497\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1498\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4185792349726776\n", + "T-Statistic: -4.200526985325906\n", + "P-Value: 0.024623895827933763\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.4048728813559322\n", + "T-Statistic: -5.068605007676937\n", + "P-Value: 0.014827407662859303\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1498\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1499\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3777322404371585\n", + "T-Statistic: -3.6747652949491756\n", + "P-Value: 0.034884091484984066\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3425612052730697\n", + "T-Statistic: -2.4888108666663245\n", + "P-Value: 0.13056092060471672\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1499\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1500\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.3916666666666667\n", + "T-Statistic: -27.593419053471173\n", + "P-Value: 0.00010447312619049549\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.37146892655367236\n", + "T-Statistic: -3.9219921560235247\n", + "P-Value: 0.029486306559984257\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1500\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1501\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.41666666666666663\n", + "T-Statistic: -5.199400639874496\n", + "P-Value: 0.013822982018819301\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.35035310734463276\n", + "T-Statistic: -3.355609536673555\n", + "P-Value: 0.043875869462308204\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1501\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1502\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.46427595628415297\n", + "T-Statistic: -4.318179445331662\n", + "P-Value: 0.022881712741708818\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.36737288135593216\n", + "T-Statistic: -5.454903356437624\n", + "P-Value: 0.012103536343441582\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1502\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1503\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3237704918032787\n", + "T-Statistic: -4.892471154105429\n", + "P-Value: 0.016335808762118268\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.288135593220339\n", + "Average of Other Ratios: 0.2362994350282486\n", + "T-Statistic: -7.416033307666337\n", + "P-Value: 0.005072640837140369\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1503\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1504\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.39166666666666666\n", + "T-Statistic: -3.8686420190221638\n", + "P-Value: 0.030555105638903693\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3206920903954802\n", + "T-Statistic: -4.327561335547406\n", + "P-Value: 0.022749742422749963\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1504\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1505\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.37766393442622953\n", + "T-Statistic: -3.0183385183635814\n", + "P-Value: 0.05683401011881442\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.29957627118644065\n", + "T-Statistic: -7.052288308361696\n", + "P-Value: 0.005860100906942432\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1505\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1506\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4062158469945355\n", + "T-Statistic: -3.358719055760332\n", + "P-Value: 0.043774850277544984\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3422316384180791\n", + "T-Statistic: -2.414469415876127\n", + "P-Value: 0.09463683116855957\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1506\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1507\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.43333333333333335\n", + "T-Statistic: -6.89052992289735\n", + "P-Value: 0.006262188755552419\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.354590395480226\n", + "T-Statistic: -6.260853478619376\n", + "P-Value: 0.008223481825561816\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1507\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1508\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.42083333333333334\n", + "T-Statistic: -8.914754098360653\n", + "P-Value: 0.00297722406530442\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3882062146892655\n", + "T-Statistic: -4.699364429896171\n", + "P-Value: 0.018227552356258447\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1508\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1509\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.3416666666666666\n", + "T-Statistic: -7.603986987982712\n", + "P-Value: 0.004720052083609356\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.31624293785310736\n", + "T-Statistic: -3.384433655570972\n", + "P-Value: 0.042950673824966874\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1509\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1510\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3691939890710383\n", + "T-Statistic: -1.986395773541501\n", + "P-Value: 0.14117716016377596\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.31235875706214694\n", + "T-Statistic: -8.330919086612344\n", + "P-Value: 0.003625030826573087\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1510\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1511\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.360724043715847\n", + "T-Statistic: -4.946082903302941\n", + "P-Value: 0.01585632915848264\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.28700564971751413\n", + "T-Statistic: -14.665257155667614\n", + "P-Value: 0.0006876691247397437\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1511\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1512\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.36666666666666664\n", + "T-Statistic: -4.588689352245944\n", + "P-Value: 0.019441198543417786\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3415960451977401\n", + "T-Statistic: -3.006477663714325\n", + "P-Value: 0.05737223684387089\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1512\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1513\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.3375\n", + "T-Statistic: -12.748930752673918\n", + "P-Value: 0.0010411491892625435\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.31193502824858754\n", + "T-Statistic: -2.8294238473612916\n", + "P-Value: 0.06622113261732844\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1513\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1514\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.32499999999999996\n", + "T-Statistic: -12.2541580554229\n", + "P-Value: 0.0011703222583523416\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3601694915254237\n", + "T-Statistic: -3.233808333817773\n", + "P-Value: 0.0480764627944047\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1514\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1515\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.3778005464480874\n", + "T-Statistic: -4.797285861708767\n", + "P-Value: 0.017234618038032837\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.35593220338983056\n", + "T-Statistic: -4.438525973846813\n", + "P-Value: 0.021261047550038326\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1515\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1516\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7666666666666667\n", + "Average of Other Ratios: 0.34405737704918027\n", + "T-Statistic: -14.172745497720056\n", + "P-Value: 0.0007609909456212724\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3432203389830508\n", + "T-Statistic: -2.8168388126916613\n", + "P-Value: 0.06691301311594693\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1516\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1517\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -5.3888664006939155\n", + "P-Value: 0.012519846650421048\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3334745762711865\n", + "T-Statistic: -3.3752877168731676\n", + "P-Value: 0.043241533194103954\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1517\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1518\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -8.518263011092074\n", + "P-Value: 0.003398441269028304\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.4176553672316384\n", + "T-Statistic: -2.608655045302455\n", + "P-Value: 0.07977746445984121\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1518\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1519\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.3458333333333333\n", + "T-Statistic: -9.713343067748339\n", + "P-Value: 0.002317591752507357\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3330508474576271\n", + "T-Statistic: -3.8022134355217396\n", + "P-Value: 0.031956488835427684\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1519\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1520\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.30833333333333335\n", + "T-Statistic: -6.506748841466283\n", + "P-Value: 0.007372760445320987\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.26581920903954803\n", + "T-Statistic: -6.147383047873269\n", + "P-Value: 0.008659663805844972\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1520\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1521\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.42083333333333334\n", + "T-Statistic: -2.8626986790948408\n", + "P-Value: 0.06443431083077424\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.3882062146892655\n", + "T-Statistic: -7.465956735313377\n", + "P-Value: 0.004975689184389457\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1521\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1522\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4065573770491804\n", + "T-Statistic: -6.594241817524086\n", + "P-Value: 0.007098081515958769\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3728813559322034\n", + "T-Statistic: -2.4230728530694363\n", + "P-Value: 0.09391016538665153\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1522\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1523\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.34419398907103826\n", + "T-Statistic: -9.378588970336171\n", + "P-Value: 0.0025678094646998427\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.31207627118644066\n", + "T-Statistic: -2.1523998630107477\n", + "P-Value: 0.12043504718327508\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1523\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1524\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3485655737704918\n", + "T-Statistic: -3.9566860100053627\n", + "P-Value: 0.028816950912038987\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.30856873822975517\n", + "T-Statistic: -2.102946127279898\n", + "P-Value: 0.1701873208222798\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1524\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1525\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3726775956284153\n", + "T-Statistic: -3.463270042711585\n", + "P-Value: 0.040543787476660896\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.2913135593220339\n", + "T-Statistic: -4.4679931695295805\n", + "P-Value: 0.020887046587591118\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1525\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1526\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.43586065573770494\n", + "T-Statistic: -4.117687550148067\n", + "P-Value: 0.02595439887765405\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3686440677966102\n", + "T-Statistic: -5.413333333333333\n", + "P-Value: 0.012363451013336186\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1526\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1527\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.42083333333333334\n", + "T-Statistic: -2.7244712370027178\n", + "P-Value: 0.07227644833071428\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3516949152542373\n", + "T-Statistic: -3.2276227144907716\n", + "P-Value: 0.048303103292659674\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1527\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1528\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.37315573770491806\n", + "T-Statistic: -7.357526587712183\n", + "P-Value: 0.005189458410858861\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3209745762711864\n", + "T-Statistic: -3.5295270186824976\n", + "P-Value: 0.038652315352543654\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1528\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1529\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.819672131147541\n", + "Average of Other Ratios: 0.32083333333333336\n", + "T-Statistic: -12.69043363555155\n", + "P-Value: 0.0010554010131434192\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3119350282485876\n", + "T-Statistic: -2.259178910141389\n", + "P-Value: 0.10900865631385685\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1529\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1530\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3372495446265938\n", + "T-Statistic: -4.51546348372934\n", + "P-Value: 0.04570861042365545\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.326271186440678\n", + "T-Statistic: -5.6348858004839935\n", + "P-Value: 0.011057151074446126\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1530\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1531\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -3.2106696051862316\n", + "P-Value: 0.04893122743058373\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.2870762711864407\n", + "T-Statistic: -5.605795079572491\n", + "P-Value: 0.011218029506615877\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1531\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1532\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.3875\n", + "T-Statistic: -2.707548854683142\n", + "P-Value: 0.07331622631866143\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.3502824858757062\n", + "T-Statistic: -4.93323850074808\n", + "P-Value: 0.015969516750107843\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1532\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1533\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.39398907103825137\n", + "T-Statistic: -3.677608363112817\n", + "P-Value: 0.03481512208257183\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3292372881355932\n", + "T-Statistic: -1.9315505529107941\n", + "P-Value: 0.14893966449897095\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1533\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1534\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4894808743169399\n", + "T-Statistic: -2.350517220396054\n", + "P-Value: 0.1002587000234438\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4470338983050848\n", + "T-Statistic: -2.370301890762351\n", + "P-Value: 0.0984770870147508\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1534\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1535\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4025273224043716\n", + "T-Statistic: -3.2715037607881903\n", + "P-Value: 0.04672400543382343\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.33340395480225987\n", + "T-Statistic: -2.965463727554792\n", + "P-Value: 0.0592833948188603\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1535\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1536\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.36441256830601093\n", + "T-Statistic: -3.6162952233041508\n", + "P-Value: 0.036342061007907864\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3347457627118644\n", + "T-Statistic: -3.0800000000000014\n", + "P-Value: 0.05413592185295247\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1536\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1537\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.4438524590163935\n", + "T-Statistic: -17.78543820658479\n", + "P-Value: 0.0003875755523465665\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.4044256120527307\n", + "T-Statistic: -2.4634512493170466\n", + "P-Value: 0.13274843347385198\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1537\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1538\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.3333333333333333\n", + "T-Statistic: -5.158898896581134\n", + "P-Value: 0.014124257169801814\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.326271186440678\n", + "T-Statistic: -2.8103535287436237\n", + "P-Value: 0.0672730683623227\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1538\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1539\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.49760928961748635\n", + "T-Statistic: -3.887955865194524\n", + "P-Value: 0.030162518753766708\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.41949152542372886\n", + "T-Statistic: -2.4681862514695623\n", + "P-Value: 0.09020943675168575\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1539\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1540\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.4562841530054645\n", + "T-Statistic: -9.590620075287022\n", + "P-Value: 0.0024054083955540014\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.423728813559322\n", + "T-Statistic: -4.404164695679157\n", + "P-Value: 0.021708194051045772\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1540\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1541\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -3.897367233401582\n", + "P-Value: 0.029973563805789114\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.40091807909604515\n", + "T-Statistic: -8.284302459961115\n", + "P-Value: 0.0036844984307791845\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1541\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1542\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -6.306001458450521\n", + "P-Value: 0.008057963024016634\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3587570621468926\n", + "T-Statistic: -3.775478418443893\n", + "P-Value: 0.032543731166538116\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1542\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1543\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.40628415300546444\n", + "T-Statistic: -2.0963572160561217\n", + "P-Value: 0.1270050338262828\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.35854519774011306\n", + "T-Statistic: -1.7390744938729776\n", + "P-Value: 0.18040454692234978\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1543\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1544\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.40416666666666673\n", + "T-Statistic: -2.1561984129969214\n", + "P-Value: 0.12000461051054315\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.35021186440677965\n", + "T-Statistic: -3.0253320224395073\n", + "P-Value: 0.05651962666233991\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1544\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1545\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4229508196721311\n", + "T-Statistic: -4.136976592067392\n", + "P-Value: 0.025636466240534804\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3923022598870057\n", + "T-Statistic: -2.2432895898859275\n", + "P-Value: 0.11062363137762433\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1545\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1546\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.42691256830601093\n", + "T-Statistic: -3.084110518603815\n", + "P-Value: 0.053961847840379426\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3714689265536723\n", + "T-Statistic: -5.879572942861632\n", + "P-Value: 0.009816556345478428\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1546\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1547\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -4.84204603440025\n", + "P-Value: 0.016804152704825513\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3338276836158192\n", + "T-Statistic: -3.2632296876876485\n", + "P-Value: 0.047016699886531045\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1547\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1548\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4023907103825136\n", + "T-Statistic: -6.903695753068684\n", + "P-Value: 0.006228128127498301\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.35169491525423724\n", + "T-Statistic: -12.115804854516572\n", + "P-Value: 0.0012102202443320875\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1548\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1549\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -22.322514291558807\n", + "P-Value: 0.00019683968706694644\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3627824858757063\n", + "T-Statistic: -8.098378653528943\n", + "P-Value: 0.003934927255862458\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1549\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1550\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.32586520947176684\n", + "T-Statistic: -3.2146877218511154\n", + "P-Value: 0.08465882990859916\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.32062146892655363\n", + "T-Statistic: -9.054535625012225\n", + "P-Value: 0.002845269537083193\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1550\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1551\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.38995901639344266\n", + "T-Statistic: -19.23389711083783\n", + "P-Value: 0.0003069437053761336\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3686440677966102\n", + "T-Statistic: -6.772746506262815\n", + "P-Value: 0.006578095712086025\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1551\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1552\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.44815573770491807\n", + "T-Statistic: -12.270797400644883\n", + "P-Value: 0.0011656421347428028\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.40925141242937857\n", + "T-Statistic: -4.746377851626824\n", + "P-Value: 0.01774180468795205\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1552\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1553\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -3.3590074850087963\n", + "P-Value: 0.043765495024786334\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3305084745762712\n", + "T-Statistic: -5.565656594987991\n", + "P-Value: 0.011445072212549055\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1553\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1554\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.3208333333333333\n", + "T-Statistic: -8.192450020709705\n", + "P-Value: 0.003805499272772318\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.3220338983050847\n", + "T-Statistic: -2.557369525291349\n", + "P-Value: 0.08340507155260364\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1554\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1555\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.5062841530054645\n", + "T-Statistic: -5.14902114352389\n", + "P-Value: 0.01419903025134191\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.40557909604519776\n", + "T-Statistic: -3.3491158311330995\n", + "P-Value: 0.044087787022349285\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1555\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1556\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.38545081967213113\n", + "T-Statistic: -6.522157996146436\n", + "P-Value: 0.007323382596367344\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.34576271186440677\n", + "T-Statistic: -6.516946235415337\n", + "P-Value: 0.007340034612930771\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1556\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1557\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.3625\n", + "T-Statistic: -5.036188941644216\n", + "P-Value: 0.015090956088189262\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.28255649717514125\n", + "T-Statistic: -2.8447438754418775\n", + "P-Value: 0.06539088569746056\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1557\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1558\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.38333333333333336\n", + "T-Statistic: -4.5941415869804265\n", + "P-Value: 0.019378995099920374\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3204802259887006\n", + "T-Statistic: -2.685797986127019\n", + "P-Value: 0.0746800724946613\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1558\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1559\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.46045081967213114\n", + "T-Statistic: -2.451392373266136\n", + "P-Value: 0.09156587040440396\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3375706214689266\n", + "T-Statistic: -3.7567808109943908\n", + "P-Value: 0.03296266830646462\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1559\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1560\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -3.3807925592909616\n", + "P-Value: 0.04306616942393178\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3309792843691149\n", + "T-Statistic: -1.7858245155746162\n", + "P-Value: 0.2160472917948211\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1560\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1561\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.49583333333333335\n", + "T-Statistic: -2.6281319368691185\n", + "P-Value: 0.07845092410121779\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.4471045197740113\n", + "T-Statistic: -3.0677087606277684\n", + "P-Value: 0.054660670246442895\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1561\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1562\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.38551912568306007\n", + "T-Statistic: -7.359999679334292\n", + "P-Value: 0.0051844490931802715\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.35918079096045197\n", + "T-Statistic: -2.275676500715352\n", + "P-Value: 0.10736186186928126\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1562\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1563\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.31905737704918036\n", + "T-Statistic: -9.174911614798257\n", + "P-Value: 0.002737770634718071\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.29936440677966103\n", + "T-Statistic: -9.423964347365773\n", + "P-Value: 0.0025318610514215717\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1563\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1564\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -3.0623788546160213\n", + "P-Value: 0.054890208233557496\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.35847457627118645\n", + "T-Statistic: -3.2665066947432746\n", + "P-Value: 0.04690049922235536\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1564\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1565\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.43041894353369764\n", + "T-Statistic: -1.6922369290882864\n", + "P-Value: 0.23267507692719033\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3969632768361582\n", + "T-Statistic: -2.896194824692244\n", + "P-Value: 0.06269598939057865\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1565\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1566\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.36489071038251364\n", + "T-Statistic: -7.458228376445201\n", + "P-Value: 0.004990536830131611\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.3177966101694915\n", + "T-Statistic: -7.600000000000018\n", + "P-Value: 0.004727189949278468\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1566\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1567\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.39842896174863385\n", + "T-Statistic: -14.529811045035604\n", + "P-Value: 0.000706861443434645\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.364406779661017\n", + "T-Statistic: -1.7643529058752923\n", + "P-Value: 0.17586309782892548\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1567\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1568\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.3861338797814208\n", + "T-Statistic: -3.095250278326531\n", + "P-Value: 0.05349362654863495\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.34152542372881356\n", + "T-Statistic: -2.7297642578116172\n", + "P-Value: 0.07195498560460242\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1568\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1569\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.3375\n", + "T-Statistic: -8.553875315773226\n", + "P-Value: 0.003357503807386855\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3419491525423729\n", + "T-Statistic: -2.59993484464519\n", + "P-Value: 0.08038034833840463\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1569\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1570\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4355191256830601\n", + "T-Statistic: -7.054729326958077\n", + "P-Value: 0.0058542974670971835\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.38834745762711864\n", + "T-Statistic: -7.704320406910871\n", + "P-Value: 0.00454501569408191\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1570\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1571\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.39583333333333326\n", + "T-Statistic: -4.287445285495584\n", + "P-Value: 0.023321028120091257\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.32033898305084746\n", + "T-Statistic: -3.891547782585247\n", + "P-Value: 0.06013719541915213\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1571\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1572\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.38989071038251366\n", + "T-Statistic: -11.606555225890991\n", + "P-Value: 0.0013736462375547754\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3543785310734463\n", + "T-Statistic: -5.771672200284073\n", + "P-Value: 0.010339886184267465\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1572\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1573\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.40211748633879785\n", + "T-Statistic: -3.9527199474578647\n", + "P-Value: 0.028892473194063042\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3474576271186441\n", + "T-Statistic: -4.802499349297195\n", + "P-Value: 0.017183746707078684\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1573\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1574\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.3875\n", + "T-Statistic: -2.3837761964250683\n", + "P-Value: 0.09728573796120057\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3504237288135593\n", + "T-Statistic: -3.262064802178429\n", + "P-Value: 0.04705809381582309\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1574\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1575\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.3958333333333333\n", + "T-Statistic: -5.180131981040665\n", + "P-Value: 0.013965256620241263\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.34611581920903955\n", + "T-Statistic: -2.3420605578207687\n", + "P-Value: 0.10103215816449211\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1575\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1576\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.48750000000000004\n", + "T-Statistic: -5.206088992974237\n", + "P-Value: 0.01377403946898648\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.4430790960451977\n", + "T-Statistic: -12.700432204420535\n", + "P-Value: 0.001052946749526643\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1576\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1577\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3984289617486339\n", + "T-Statistic: -3.252508058107846\n", + "P-Value: 0.047399438327312575\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.36292372881355933\n", + "T-Statistic: -5.327313464522444\n", + "P-Value: 0.012924898405343976\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1577\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1578\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.45833333333333326\n", + "T-Statistic: -3.755595353518015\n", + "P-Value: 0.032989461923480595\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.38425141242937855\n", + "T-Statistic: -4.698424427455757\n", + "P-Value: 0.018237439696403804\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1578\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1579\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.43333333333333335\n", + "T-Statistic: -5.352084912420385\n", + "P-Value: 0.012759859239265056\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3882768361581921\n", + "T-Statistic: -6.446590136835602\n", + "P-Value: 0.007569780791386365\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1579\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1580\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3859289617486339\n", + "T-Statistic: -6.327570523296936\n", + "P-Value: 0.007980431881706069\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.328954802259887\n", + "T-Statistic: -7.995948131756914\n", + "P-Value: 0.004082555066940009\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1580\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1581\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.3583333333333333\n", + "T-Statistic: -8.261956373376224\n", + "P-Value: 0.0037134622752403278\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3249293785310734\n", + "T-Statistic: -4.059276987931347\n", + "P-Value: 0.026948683537732478\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1581\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1582\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.39583333333333337\n", + "T-Statistic: -14.806483893669949\n", + "P-Value: 0.0006683885463905405\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3601694915254237\n", + "T-Statistic: -4.444444444444445\n", + "P-Value: 0.0211852376832342\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1582\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1583\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -6.0055128469177514\n", + "P-Value: 0.009248776088665693\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3135593220338983\n", + "T-Statistic: -6.483004673258841\n", + "P-Value: 0.007449708002644034\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1583\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1584\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7333333333333333\n", + "Average of Other Ratios: 0.3855874316939891\n", + "T-Statistic: -13.52760363997253\n", + "P-Value: 0.0008736345306870901\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3586158192090395\n", + "T-Statistic: -3.921613238217157\n", + "P-Value: 0.029493726846163477\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1584\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1585\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -5.923439030264504\n", + "P-Value: 0.009613733901387398\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3038135593220339\n", + "T-Statistic: -9.848354363498629\n", + "P-Value: 0.0022258218489771527\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1585\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1586\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.32499999999999996\n", + "T-Statistic: -15.651197399435246\n", + "P-Value: 0.0005668681892135681\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.30819209039548023\n", + "T-Statistic: -2.8185816226035203\n", + "P-Value: 0.06681666397008225\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1586\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1587\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.3041666666666667\n", + "T-Statistic: -10.064072122074942\n", + "P-Value: 0.0020889346595399047\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3077683615819209\n", + "T-Statistic: -4.319796363784691\n", + "P-Value: 0.02285889788909435\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1587\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1588\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.3625\n", + "T-Statistic: -4.290763799723821\n", + "P-Value: 0.02327307093304941\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.3008474576271187\n", + "T-Statistic: -3.7673867088977024\n", + "P-Value: 0.032724190890523516\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1588\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1589\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -7.686877814143884\n", + "P-Value: 0.004574822356362937\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.32909604519774016\n", + "T-Statistic: -5.006421579143373\n", + "P-Value: 0.015338372293344877\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1589\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1590\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.24057377049180326\n", + "T-Statistic: -10.793811013450267\n", + "P-Value: 0.0017009322854444507\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.26666666666666666\n", + "Average of Other Ratios: 0.1822033898305085\n", + "T-Statistic: -10.409843826150722\n", + "P-Value: 0.001891887685589342\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1590\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1591\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.40635245901639344\n", + "T-Statistic: -3.8261690084495177\n", + "P-Value: 0.03144179376080752\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.32895480225988705\n", + "T-Statistic: -2.949093002418226\n", + "P-Value: 0.060068491192589654\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1591\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1592\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4396857923497268\n", + "T-Statistic: -6.979319263082819\n", + "P-Value: 0.006037138823969032\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.4213747645951036\n", + "T-Statistic: -3.60190807688241\n", + "P-Value: 0.06917625132966015\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1592\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1593\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -5.646420206490955\n", + "P-Value: 0.010994199496502145\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3712570621468927\n", + "T-Statistic: -2.1556241033270016\n", + "P-Value: 0.12006957075894573\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1593\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1594\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.35724043715847\n", + "T-Statistic: -2.2370139019506667\n", + "P-Value: 0.11126942224811241\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3081214689265537\n", + "T-Statistic: -3.8998685439923104\n", + "P-Value: 0.029923600161968773\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1594\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1595\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4478825136612022\n", + "T-Statistic: -5.2484139227280515\n", + "P-Value: 0.01346950981253263\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3206214689265537\n", + "T-Statistic: -3.0461946335774264\n", + "P-Value: 0.05559466924951487\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1595\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1596\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.4541666666666667\n", + "T-Statistic: -3.811215023955327\n", + "P-Value: 0.03176182755813695\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3803672316384181\n", + "T-Statistic: -2.5550917784808758\n", + "P-Value: 0.08357084637244536\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1596\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1597\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3770491803278689\n", + "T-Statistic: -5.292311191396577\n", + "P-Value: 0.013162886761926772\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3311676082862524\n", + "T-Statistic: -2.2490974729241855\n", + "P-Value: 0.1534474328108915\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1597\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1598\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8333333333333334\n", + "Average of Other Ratios: 0.4108606557377049\n", + "T-Statistic: -28.554907520499913\n", + "P-Value: 9.430057361661532e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3293079096045198\n", + "T-Statistic: -8.10248857720817\n", + "P-Value: 0.003929152121721815\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1598\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1599\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4064207650273224\n", + "T-Statistic: -5.716648520223962\n", + "P-Value: 0.01062086627428458\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.3432203389830508\n", + "T-Statistic: -5.53333333333333\n", + "P-Value: 0.011632297103592594\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1599\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1600\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4354508196721312\n", + "T-Statistic: -6.567850831861847\n", + "P-Value: 0.007179495661119332\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.401271186440678\n", + "T-Statistic: -2.459129645217466\n", + "P-Value: 0.09093785730195074\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1600\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1601\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4625\n", + "T-Statistic: -4.525936567821964\n", + "P-Value: 0.02017613595003384\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.4557909604519774\n", + "T-Statistic: -3.177872547530921\n", + "P-Value: 0.05017596813649382\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1601\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1602\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.2616120218579235\n", + "T-Statistic: -5.789061661601357\n", + "P-Value: 0.010253114813387197\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.23728813559322035\n", + "Average of Other Ratios: 0.19837570621468925\n", + "T-Statistic: -4.553719008264464\n", + "P-Value: 0.01984639027416479\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1602\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1603\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.4208333333333333\n", + "T-Statistic: -4.46482463507427\n", + "P-Value: 0.020926850914564546\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.39279661016949147\n", + "T-Statistic: -2.2752115860184423\n", + "P-Value: 0.10740785666680364\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1603\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1604\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.373292349726776\n", + "T-Statistic: -5.837995555994575\n", + "P-Value: 0.010014006051520334\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3667372881355932\n", + "T-Statistic: -2.5590817041221827\n", + "P-Value: 0.08328072516772207\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1604\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1605\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.32083333333333336\n", + "T-Statistic: -5.409452966923079\n", + "P-Value: 0.012388083734135366\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.307909604519774\n", + "T-Statistic: -4.795831523312723\n", + "P-Value: 0.017248843874307026\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1605\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1606\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -13.136517577876024\n", + "P-Value: 0.0009528915088720033\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.39646892655367233\n", + "T-Statistic: -2.0743037811227554\n", + "P-Value: 0.12970694368573776\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1606\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1607\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.38565573770491807\n", + "T-Statistic: -1.9388846252338177\n", + "P-Value: 0.1478731519159202\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.29199623352165727\n", + "T-Statistic: -4.218647880820993\n", + "P-Value: 0.05185732044957473\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1607\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1608\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.33975409836065573\n", + "T-Statistic: -7.109185568026944\n", + "P-Value: 0.005726775618868671\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.307909604519774\n", + "T-Statistic: -3.8877095717511785\n", + "P-Value: 0.030167484179746327\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1608\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1609\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4028005464480875\n", + "T-Statistic: -4.245647176969608\n", + "P-Value: 0.02393615072729287\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3389830508474576\n", + "T-Statistic: -3.1281966106828594\n", + "P-Value: 0.05213847018410804\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1609\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1610\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.31509562841530053\n", + "T-Statistic: -8.1350398352272\n", + "P-Value: 0.003883805394600376\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.2911016949152542\n", + "T-Statistic: -3.8194407004959707\n", + "P-Value: 0.03158527352615677\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1610\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1611\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.275\n", + "T-Statistic: -8.547594094887065\n", + "P-Value: 0.0033646767112774094\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.2796610169491525\n", + "T-Statistic: -3.422222222222223\n", + "P-Value: 0.041774946157591517\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1611\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1612\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.37739071038251365\n", + "T-Statistic: -4.570466183924919\n", + "P-Value: 0.019650992181636566\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.3008474576271186\n", + "T-Statistic: -3.396132253860897\n", + "P-Value: 0.042582247398141884\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1612\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1613\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.38749999999999996\n", + "T-Statistic: -10.334980525866824\n", + "P-Value: 0.001932389182639011\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4008474576271187\n", + "T-Statistic: -5.126174517160044\n", + "P-Value: 0.01437396472653316\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1613\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1614\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.46461748633879785\n", + "T-Statistic: -6.40281942286815\n", + "P-Value: 0.007717499991952441\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4093926553672317\n", + "T-Statistic: -3.8752447538675496\n", + "P-Value: 0.030420159550251232\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1614\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1615\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.39166666666666666\n", + "T-Statistic: -4.3147109239249755\n", + "P-Value: 0.022930753286261416\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3220338983050848\n", + "T-Statistic: -2.456339176607667\n", + "P-Value: 0.0911637428376164\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1615\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1616\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3690573770491803\n", + "T-Statistic: -3.785362525900684\n", + "P-Value: 0.0323250246243217\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.28721751412429375\n", + "T-Statistic: -3.7810435820759127\n", + "P-Value: 0.03242035778750225\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1616\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1617\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4151639344262295\n", + "T-Statistic: -3.7566753428966173\n", + "P-Value: 0.03296505095792107\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3291666666666667\n", + "T-Statistic: -4.334673395356118\n", + "P-Value: 0.022650355990863152\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1617\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1618\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.35416666666666663\n", + "T-Statistic: -17.816045884784717\n", + "P-Value: 0.0003855963684750506\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.2828389830508475\n", + "T-Statistic: -10.598274792221071\n", + "P-Value: 0.0017948089853278408\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1618\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1619\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.35239071038251363\n", + "T-Statistic: -4.290395689342995\n", + "P-Value: 0.023278384354672565\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3050847457627119\n", + "T-Statistic: -2.1744034361245617\n", + "P-Value: 0.1179670753099225\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1619\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1620\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8032786885245902\n", + "Average of Other Ratios: 0.4291666666666667\n", + "T-Statistic: -17.0700379678159\n", + "P-Value: 0.00043795305550041073\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6271186440677966\n", + "Average of Other Ratios: 0.4050847457627119\n", + "T-Statistic: -9.240027567747635\n", + "P-Value: 0.0026818554468391116\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1620\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1621\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -27.264182776241938\n", + "P-Value: 0.00010829140307033679\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3543785310734463\n", + "T-Statistic: -7.613153050047849\n", + "P-Value: 0.004703696031017581\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1621\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1622\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.4166666666666667\n", + "T-Statistic: -3.172289994751985\n", + "P-Value: 0.05039180926567533\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3940677966101695\n", + "T-Statistic: -2.976518270891203\n", + "P-Value: 0.05876051507230344\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1622\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1623\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3867030965391621\n", + "T-Statistic: -18.78378378378386\n", + "P-Value: 0.0028222292471545055\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.38411016949152543\n", + "T-Statistic: -7.506646894907283\n", + "P-Value: 0.0048984693502791745\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1623\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1624\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -6.460980283821876\n", + "P-Value: 0.007522028995284054\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.39668079096045195\n", + "T-Statistic: -3.674203780886493\n", + "P-Value: 0.034897733800121254\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1624\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1625\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -8.291518233493496\n", + "P-Value: 0.0036752095251250446\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3627824858757062\n", + "T-Statistic: -5.044529690751584\n", + "P-Value: 0.015022564694235147\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1625\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1626\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.38524590163934425\n", + "T-Statistic: -2.953558943086598\n", + "P-Value: 0.059853030783475765\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.32083333333333336\n", + "T-Statistic: -5.361374140743174\n", + "P-Value: 0.012698680109057153\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1626\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1627\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4278005464480874\n", + "T-Statistic: -3.220215216567121\n", + "P-Value: 0.048576294366246366\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.371045197740113\n", + "T-Statistic: -5.171216742476558\n", + "P-Value: 0.014031731053501115\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1627\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1628\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.35416666666666663\n", + "T-Statistic: -7.776108221599537\n", + "P-Value: 0.00442500536007881\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.3347457627118644\n", + "T-Statistic: -3.822222222222226\n", + "P-Value: 0.03152585651673634\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1628\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1629\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.3375\n", + "T-Statistic: -6.2978017747795265\n", + "P-Value: 0.00808769678640319\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.3038135593220339\n", + "T-Statistic: -27.666666666666476\n", + "P-Value: 0.00010364812386939461\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1629\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1630\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.39392076502732243\n", + "T-Statistic: -2.8228292898095364\n", + "P-Value: 0.06658256000513207\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3425612052730697\n", + "T-Statistic: -7.542807340088425\n", + "P-Value: 0.01712633052558875\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1630\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1631\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.3321038251366121\n", + "T-Statistic: -9.748782925528516\n", + "P-Value: 0.002293024514092687\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.2867937853107344\n", + "T-Statistic: -3.780359197747885\n", + "P-Value: 0.032435497445757994\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1631\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1632\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3729508196721311\n", + "T-Statistic: -3.16062146897757\n", + "P-Value: 0.05084675449389502\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.35430790960451974\n", + "T-Statistic: -2.986076879787708\n", + "P-Value: 0.058313047921487345\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1632\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1633\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.31974043715846995\n", + "T-Statistic: -16.761645970184002\n", + "P-Value: 0.0004623636950420215\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3220338983050847\n", + "Average of Other Ratios: 0.2911016949152543\n", + "T-Statistic: -2.5053535141492107\n", + "P-Value: 0.08729389266327836\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1633\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1634\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4396174863387978\n", + "T-Statistic: -7.124258314479804\n", + "P-Value: 0.0056921272252719556\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.38389830508474576\n", + "T-Statistic: -3.5352476162925313\n", + "P-Value: 0.03849432965830513\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1634\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1635\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.46072404371584696\n", + "T-Statistic: -5.791146439440572\n", + "P-Value: 0.010242776156530948\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3296610169491525\n", + "T-Statistic: -2.932194632545476\n", + "P-Value: 0.060892602743676055\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1635\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1636\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.4125\n", + "T-Statistic: -7.478844437595781\n", + "P-Value: 0.00495105882203386\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3145951035781544\n", + "T-Statistic: -5.112873614515615\n", + "P-Value: 0.036189580961726336\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1636\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1637\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.4194672131147541\n", + "T-Statistic: -3.705356086132572\n", + "P-Value: 0.0341510693846338\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.388135593220339\n", + "T-Statistic: -3.316624790355395\n", + "P-Value: 0.04516783583455813\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1637\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1638\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.2583333333333333\n", + "T-Statistic: -11.303389666068615\n", + "P-Value: 0.0014850505321337656\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.24039548022598872\n", + "T-Statistic: -2.2432542767385697\n", + "P-Value: 0.1106272525728738\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1638\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1639\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4262295081967213\n", + "Average of Other Ratios: 0.37499999999999994\n", + "T-Statistic: -2.582483761527067\n", + "P-Value: 0.0816038163981699\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.2913135593220339\n", + "T-Statistic: -4.63515148563193\n", + "P-Value: 0.01891930032646319\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1639\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1640\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.3319672131147541\n", + "T-Statistic: -16.69265269416492\n", + "P-Value: 0.00046807129563190304\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.31645480225988704\n", + "T-Statistic: -7.447420720142193\n", + "P-Value: 0.005011398574053057\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1640\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1641\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.42718579234972676\n", + "T-Statistic: -4.989229131610327\n", + "P-Value: 0.01548368101755126\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.4025423728813559\n", + "T-Statistic: -4.372697328618899\n", + "P-Value: 0.02212841807396774\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1641\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1642\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -11.48552138205233\n", + "P-Value: 0.0014167460006659608\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.36525423728813555\n", + "T-Statistic: -9.888888888888882\n", + "P-Value: 0.010071751888174055\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1642\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1643\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3734289617486339\n", + "T-Statistic: -7.373424834781318\n", + "P-Value: 0.0051573670428228195\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.31631355932203387\n", + "T-Statistic: -4.212334695138946\n", + "P-Value: 0.024441477588059136\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1643\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1644\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8\n", + "Average of Other Ratios: 0.3737704918032787\n", + "T-Statistic: -11.276355501471617\n", + "P-Value: 0.0014955591121438806\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3838983050847458\n", + "T-Statistic: -13.55263062255603\n", + "P-Value: 0.0008688656783751102\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1644\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1645\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -6.697501094028932\n", + "P-Value: 0.006790960911751886\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.29583333333333334\n", + "T-Statistic: -3.9331245586533203\n", + "P-Value: 0.029269369284174996\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1645\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1646\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -18.208323627746466\n", + "P-Value: 0.0003613808590242596\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3418079096045198\n", + "T-Statistic: -5.938574464184704\n", + "P-Value: 0.00954502881900154\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1646\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1647\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.47500000000000003\n", + "T-Statistic: -5.911475409836067\n", + "P-Value: 0.009668500550400335\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.42189265536723164\n", + "T-Statistic: -2.446859704851505\n", + "P-Value: 0.09193623834826871\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1647\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1648\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.35\n", + "T-Statistic: -3.567189130512097\n", + "P-Value: 0.03762717652367037\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.31242937853107344\n", + "T-Statistic: -7.44762841439633\n", + "P-Value: 0.005010996583977287\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1648\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1649\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.4062841530054645\n", + "T-Statistic: -1.9500758855993163\n", + "P-Value: 0.1462629282557643\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.35021186440677965\n", + "T-Statistic: -6.955318569856883\n", + "P-Value: 0.006096907162548966\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1649\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1650\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.3291666666666666\n", + "T-Statistic: -35.834028669908896\n", + "P-Value: 4.779336400647669e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3627824858757063\n", + "T-Statistic: -6.8705764571632155\n", + "P-Value: 0.006314278455610205\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1650\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1651\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.36475409836065575\n", + "T-Statistic: -5.115536577871306\n", + "P-Value: 0.014456377843406167\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3165254237288136\n", + "T-Statistic: -8.288366308708238\n", + "P-Value: 0.0036792632033313915\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1651\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1652\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3778005464480874\n", + "T-Statistic: -2.8771732821175253\n", + "P-Value: 0.06367584045734935\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.31666666666666665\n", + "T-Statistic: -4.566003911847601\n", + "P-Value: 0.019702811079520054\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1652\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1653\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.39064207650273225\n", + "T-Statistic: -2.5781209852352887\n", + "P-Value: 0.08191326217312117\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.364406779661017\n", + "T-Statistic: -2.9623708453941098\n", + "P-Value: 0.05943073144700672\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1653\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1654\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.398155737704918\n", + "T-Statistic: -6.811947916107325\n", + "P-Value: 0.006470670821303707\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3332627118644068\n", + "T-Statistic: -2.9486299139697687\n", + "P-Value: 0.06009088857589412\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1654\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1655\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.35416666666666663\n", + "T-Statistic: -3.722323789331516\n", + "P-Value: 0.03375297717263195\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.32040960451977407\n", + "T-Statistic: -1.8464888902941\n", + "P-Value: 0.16198897032955312\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1655\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1656\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.40225409836065573\n", + "T-Statistic: -7.458643842395928\n", + "P-Value: 0.004989737155548977\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.341454802259887\n", + "T-Statistic: -2.2689902462968896\n", + "P-Value: 0.10802564058574593\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1656\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1657\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.39583333333333337\n", + "T-Statistic: -4.726024019252636\n", + "P-Value: 0.017950011968011444\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.31242937853107344\n", + "T-Statistic: -5.102734357993427\n", + "P-Value: 0.01455637683815113\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1657\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1658\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.2833333333333333\n", + "T-Statistic: -23.253380378036304\n", + "P-Value: 0.00017423225248505516\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.2754237288135593\n", + "T-Statistic: -2.66340767777208\n", + "P-Value: 0.07611699066362566\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1658\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1659\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.4375\n", + "T-Statistic: -3.696204554145968\n", + "P-Value: 0.034368276506610014\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.35035310734463276\n", + "T-Statistic: -4.624808617471442\n", + "P-Value: 0.019033890844563105\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1659\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1660\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.41976320582877963\n", + "T-Statistic: -5.126120909780012\n", + "P-Value: 0.03601274479900024\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.37549435028248584\n", + "T-Statistic: -2.153106812065672\n", + "P-Value: 0.12035479878967444\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1660\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1661\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.31666666666666665\n", + "T-Statistic: -3.5694569453443568\n", + "P-Value: 0.03756656146901733\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2711864406779661\n", + "Average of Other Ratios: 0.24708097928436912\n", + "T-Statistic: -2.461538461538462\n", + "P-Value: 0.13291549663455224\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1661\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1662\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.31666666666666665\n", + "T-Statistic: -19.02035477866064\n", + "P-Value: 0.000317329217166385\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.30798022598870056\n", + "T-Statistic: -2.5144900114567874\n", + "P-Value: 0.08659496027337239\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1662\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1663\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.2987704918032787\n", + "T-Statistic: -9.287563234189735\n", + "P-Value: 0.002641987913158529\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.26151129943502827\n", + "T-Statistic: -8.20761557188874\n", + "P-Value: 0.0037851627982459913\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1663\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1664\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.35\n", + "T-Statistic: -20.378585595507392\n", + "P-Value: 0.00025834295408300217\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3177966101694915\n", + "T-Statistic: -3.377106525540539\n", + "P-Value: 0.043183492567890926\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1664\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1665\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.448292349726776\n", + "T-Statistic: -3.5599428053606994\n", + "P-Value: 0.03782169758880093\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.39216101694915256\n", + "T-Statistic: -4.374860785929049\n", + "P-Value: 0.02209919137309572\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1665\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1666\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4021857923497268\n", + "T-Statistic: -8.675666340422445\n", + "P-Value: 0.0032223344164688237\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3940677966101695\n", + "T-Statistic: -4.0051372633147\n", + "P-Value: 0.027914510109473\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1666\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1667\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.3958333333333333\n", + "T-Statistic: -12.228241897871557\n", + "P-Value: 0.0011776614876831963\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.30798022598870056\n", + "T-Statistic: -5.52252684001195\n", + "P-Value: 0.011695784203535333\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1667\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1668\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.4060109289617486\n", + "T-Statistic: -4.451765940749828\n", + "P-Value: 0.0210919401610888\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.41334745762711866\n", + "T-Statistic: -3.0332358222638836\n", + "P-Value: 0.05616694738868672\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1668\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1669\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.3400956284153005\n", + "T-Statistic: -16.126211408017976\n", + "P-Value: 0.0005186735594686228\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.33326271186440676\n", + "T-Statistic: -3.2380690490156394\n", + "P-Value: 0.04792113259547419\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1669\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1670\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.3583333333333334\n", + "T-Statistic: -21.740550300468346\n", + "P-Value: 0.00021299095231525587\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3247175141242938\n", + "T-Statistic: -3.864669010850272\n", + "P-Value: 0.030636676156145785\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1670\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1671\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4187158469945355\n", + "T-Statistic: -5.0760762819997485\n", + "P-Value: 0.014767517268100116\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3754943502824859\n", + "T-Statistic: -18.459459459459406\n", + "P-Value: 0.0003469320912607935\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1671\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1672\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4314890710382514\n", + "T-Statistic: -4.691101983797846\n", + "P-Value: 0.018314699259855377\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.35444915254237286\n", + "T-Statistic: -7.394856051762754\n", + "P-Value: 0.005114520056169129\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1672\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1673\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4105191256830601\n", + "T-Statistic: -6.871714846368723\n", + "P-Value: 0.006311291308420261\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.38389830508474576\n", + "T-Statistic: -2.591193878173862\n", + "P-Value: 0.08099031623314684\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1673\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1674\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4023907103825136\n", + "T-Statistic: -3.1785087830863152\n", + "P-Value: 0.050151443196549406\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3372175141242938\n", + "T-Statistic: -2.2461944693225684\n", + "P-Value: 0.11032623843400814\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1674\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1675\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.38989071038251366\n", + "T-Statistic: -8.909952627909695\n", + "P-Value: 0.002981899212734121\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.35444915254237286\n", + "T-Statistic: -69.57142857142838\n", + "P-Value: 6.544177344803568e-06\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1675\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1676\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.3488387978142076\n", + "T-Statistic: -10.158428205179094\n", + "P-Value: 0.0020325642199010164\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3347457627118644\n", + "T-Statistic: -2.262423733759691\n", + "P-Value: 0.10868236109185891\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1676\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1677\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.39016393442622954\n", + "T-Statistic: -4.155570602285484\n", + "P-Value: 0.025334728700013992\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3305084745762712\n", + "T-Statistic: -2.886707772014044\n", + "P-Value: 0.06318231943174393\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1677\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1678\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.426844262295082\n", + "T-Statistic: -3.4665463985771736\n", + "P-Value: 0.04044752053565438\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.39661016949152544\n", + "T-Statistic: -5.396227020339848\n", + "P-Value: 0.012472526064506972\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1678\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1679\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.42295081967213116\n", + "T-Statistic: -5.864413389759132\n", + "P-Value: 0.009887952398219825\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3483050847457627\n", + "T-Statistic: -11.775818566563489\n", + "P-Value: 0.00713429023847155\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1679\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1680\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4019808743169399\n", + "T-Statistic: -3.3292886989365154\n", + "P-Value: 0.044742920567822704\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3541666666666667\n", + "T-Statistic: -3.8771841553769035\n", + "P-Value: 0.030380667708462643\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1680\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1681\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.39999999999999997\n", + "T-Statistic: -9.082476164942625\n", + "P-Value: 0.0028198267013386054\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3629237288135593\n", + "T-Statistic: -4.044393553582212\n", + "P-Value: 0.02720985020655579\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1681\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1682\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.3541666666666667\n", + "T-Statistic: -5.142940376750061\n", + "P-Value: 0.014245317657868042\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.30798022598870056\n", + "T-Statistic: -2.584061622196982\n", + "P-Value: 0.08149225560483021\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1682\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1683\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.3791666666666667\n", + "T-Statistic: -10.107871974839727\n", + "P-Value: 0.0020625133025401216\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3711158192090395\n", + "T-Statistic: -5.036029713302375\n", + "P-Value: 0.015092265651518118\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1683\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1684\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.41666666666666663\n", + "T-Statistic: -15.679030900154354\n", + "P-Value: 0.0005638836832955063\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.38813559322033897\n", + "T-Statistic: -3.0606121836612235\n", + "P-Value: 0.05496655926720403\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1684\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1685\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.41666666666666663\n", + "T-Statistic: -12.515152510941379\n", + "P-Value: 0.0010996877982163017\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3965395480225989\n", + "T-Statistic: -4.332720166230066\n", + "P-Value: 0.022677595074616603\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1685\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1686\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4605191256830601\n", + "T-Statistic: -4.084201256082031\n", + "P-Value: 0.02651851352137615\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3933145009416196\n", + "T-Statistic: -6.776908139806704\n", + "P-Value: 0.021087634455750442\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1686\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1687\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4396857923497267\n", + "T-Statistic: -2.8054956411927168\n", + "P-Value: 0.06754435460228396\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.40896892655367234\n", + "T-Statistic: -2.7252378965546162\n", + "P-Value: 0.07222977633186194\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1687\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1688\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -10.239067447514469\n", + "P-Value: 0.001985976142870652\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.35854519774011295\n", + "T-Statistic: -7.385335801707823\n", + "P-Value: 0.005133495507004227\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1688\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1689\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.42083333333333334\n", + "T-Statistic: -5.382812644531547\n", + "P-Value: 0.012558941549593422\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3754943502824859\n", + "T-Statistic: -5.31399624894294\n", + "P-Value: 0.013014777489807249\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1689\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1690\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.42711748633879776\n", + "T-Statistic: -1.4944177904953027\n", + "P-Value: 0.2319280697354033\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.32881355932203393\n", + "T-Statistic: -3.1543372140892108\n", + "P-Value: 0.051093920277185956\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1690\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1691\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.40232240437158473\n", + "T-Statistic: -6.785424665478694\n", + "P-Value: 0.006543098179519793\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3628531073446328\n", + "T-Statistic: -5.382081108767228\n", + "P-Value: 0.012563676575340528\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1691\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1692\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3818989071038251\n", + "T-Statistic: -3.1559440965062726\n", + "P-Value: 0.05103057602991868\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.34745762711864403\n", + "T-Statistic: -7.849246248313698\n", + "P-Value: 0.004307003389198978\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1692\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1693\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -3.141020234264478\n", + "P-Value: 0.051622720218920067\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3163841807909604\n", + "T-Statistic: -2.724020898427997\n", + "P-Value: 0.07230388107794879\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1693\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1694\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.33706739526411655\n", + "T-Statistic: -12.955859792395307\n", + "P-Value: 0.005904832207349002\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3038135593220339\n", + "T-Statistic: -4.135972456516446\n", + "P-Value: 0.025652892626118445\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1694\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1695\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.32916666666666666\n", + "T-Statistic: -8.41751169737434\n", + "P-Value: 0.0035178920060083122\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.2786016949152542\n", + "T-Statistic: -6.564422610165478\n", + "P-Value: 0.007190161439169192\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1695\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1696\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.4541666666666666\n", + "T-Statistic: -3.061039563134522\n", + "P-Value: 0.0549480767673967\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.36864406779661013\n", + "T-Statistic: -3.3422909943494004\n", + "P-Value: 0.04431191289649832\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1696\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1697\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.39822404371584696\n", + "T-Statistic: -3.1169399358461005\n", + "P-Value: 0.052596564456531714\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.364406779661017\n", + "T-Statistic: -4.131182235954578\n", + "P-Value: 0.02573144168204107\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1697\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1698\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.425\n", + "T-Statistic: -4.093911885056293\n", + "P-Value: 0.026353313515894105\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3813559322033898\n", + "T-Statistic: -2.919201796799047\n", + "P-Value: 0.06153589112407167\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1698\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1699\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.41072404371584703\n", + "T-Statistic: -16.306113587728134\n", + "P-Value: 0.0005018457959471869\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.35444915254237286\n", + "T-Statistic: -4.447863008870363\n", + "P-Value: 0.021141608672485282\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1699\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1700\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.46249999999999997\n", + "T-Statistic: -3.08642244690056\n", + "P-Value: 0.05386425088654483\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.34611581920903955\n", + "T-Statistic: -7.952423126063747\n", + "P-Value: 0.004147501211922912\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1700\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1701\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.43155737704918035\n", + "T-Statistic: -7.41608714028745\n", + "P-Value: 0.00507253495755017\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.354590395480226\n", + "T-Statistic: -4.195847013586463\n", + "P-Value: 0.024696683385781026\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1701\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1702\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.42916666666666664\n", + "T-Statistic: -3.7914997183027954\n", + "P-Value: 0.03219017406449965\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.4514124293785311\n", + "T-Statistic: -2.7684223456685544\n", + "P-Value: 0.06966019639221771\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1702\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1703\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4023224043715847\n", + "T-Statistic: -5.3872290083001095\n", + "P-Value: 0.012530405150481858\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.38983050847457623\n", + "T-Statistic: -3.3480885546378523\n", + "P-Value: 0.04412143039369844\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1703\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1704\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3773224043715847\n", + "T-Statistic: -4.640651750707308\n", + "P-Value: 0.018858726540020358\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3332627118644068\n", + "T-Statistic: -3.6621184863223024\n", + "P-Value: 0.03519301414612658\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1704\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1705\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3860655737704918\n", + "T-Statistic: -3.4475208334165512\n", + "P-Value: 0.04101062763956584\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.33742937853107347\n", + "T-Statistic: -4.962234528821576\n", + "P-Value: 0.015715475861098616\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1705\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1706\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.4333333333333334\n", + "T-Statistic: -5.465357642986741\n", + "P-Value: 0.01203929791125337\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.40098870056497177\n", + "T-Statistic: -3.2752006464574146\n", + "P-Value: 0.04659397436436988\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1706\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1707\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.3458333333333333\n", + "T-Statistic: -3.3291936924471166\n", + "P-Value: 0.04474608942509305\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.2923728813559322\n", + "T-Statistic: -3.2984845004941277\n", + "P-Value: 0.04578545695456099\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1707\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1708\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.37315573770491806\n", + "T-Statistic: -2.5737901679797495\n", + "P-Value: 0.0822218718502907\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3333333333333333\n", + "Average of Other Ratios: 0.2754237288135593\n", + "T-Statistic: -3.609848715935058\n", + "P-Value: 0.03650753702324681\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1708\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1709\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.41666666666666663\n", + "T-Statistic: -9.248430069349528\n", + "P-Value: 0.0026747507004180183\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3813559322033898\n", + "T-Statistic: -10.066666666666663\n", + "P-Value: 0.0020873570938939115\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1709\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1710\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.3398224043715847\n", + "T-Statistic: -7.915878353618565\n", + "P-Value: 0.004203089684479934\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3220338983050848\n", + "T-Statistic: -5.039047529047529\n", + "P-Value: 0.01506747099332577\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1710\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1711\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.419603825136612\n", + "T-Statistic: -4.140759720616552\n", + "P-Value: 0.025574700755914715\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3432203389830508\n", + "T-Statistic: -3.9231182932531077\n", + "P-Value: 0.029464267753910946\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1711\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1712\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.4316939890710383\n", + "T-Statistic: -11.006210975743855\n", + "P-Value: 0.0016061976208647793\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.43644067796610164\n", + "T-Statistic: -2.3596084289526735\n", + "P-Value: 0.09943521206177618\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1712\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1713\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.38627049180327866\n", + "T-Statistic: -2.911570443803633\n", + "P-Value: 0.06191769354930334\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.3360169491525423\n", + "T-Statistic: -1.4242424242424274\n", + "P-Value: 0.38970838325942464\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1713\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1714\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.46427595628415297\n", + "T-Statistic: -1.8536473961974427\n", + "P-Value: 0.16084088720625275\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.4132768361581921\n", + "T-Statistic: -5.184819470451837\n", + "P-Value: 0.013930470217760019\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1714\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1715\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.4125\n", + "T-Statistic: -2.7434685305994506\n", + "P-Value: 0.07113088858607176\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4091101694915254\n", + "T-Statistic: -3.301274514960128\n", + "P-Value: 0.04568977195819744\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1715\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1716\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3528688524590164\n", + "T-Statistic: -4.518525749449755\n", + "P-Value: 0.02026529222371772\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.2870762711864407\n", + "T-Statistic: -8.442559461787118\n", + "P-Value: 0.003487683310568225\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1716\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1717\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4687158469945355\n", + "T-Statistic: -6.390672157378807\n", + "P-Value: 0.007759165768176083\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.40508474576271186\n", + "T-Statistic: -4.984211525609615\n", + "P-Value: 0.015526427125038359\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1717\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1718\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.39788251366120214\n", + "T-Statistic: -5.975152199128738\n", + "P-Value: 0.009381627777611113\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.40084745762711865\n", + "T-Statistic: -9.945373183606224\n", + "P-Value: 0.00216282978388815\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1718\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1719\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.44583333333333336\n", + "T-Statistic: -4.1303709900535255\n", + "P-Value: 0.025744775027266065\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3882062146892655\n", + "T-Statistic: -4.699364429896171\n", + "P-Value: 0.018227552356258447\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1719\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1720\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.32766393442622954\n", + "T-Statistic: -15.565235673998512\n", + "P-Value: 0.0005762193133083165\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.28700564971751413\n", + "T-Statistic: -4.0757619541037435\n", + "P-Value: 0.02666317079369889\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1720\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1721\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.39166666666666666\n", + "T-Statistic: -6.397744904460955\n", + "P-Value: 0.007734870011183786\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.358545197740113\n", + "T-Statistic: -5.255556820568775\n", + "P-Value: 0.01341898591418548\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1721\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1722\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.41666666666666663\n", + "T-Statistic: -2.647817826363915\n", + "P-Value: 0.0771377198138251\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.34187853107344635\n", + "T-Statistic: -4.739270182928911\n", + "P-Value: 0.017814153415116545\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1722\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1723\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.43128415300546447\n", + "T-Statistic: -3.57266399939355\n", + "P-Value: 0.03748105460475532\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.36299435028248583\n", + "T-Statistic: -4.303404214552464\n", + "P-Value: 0.02309156259614923\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1723\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1724\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.4625\n", + "T-Statistic: -2.7495312426784695\n", + "P-Value: 0.07077005262877357\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.4049435028248588\n", + "T-Statistic: -2.885264224008987\n", + "P-Value: 0.06325673205698319\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1724\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1725\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.3291666666666666\n", + "T-Statistic: -16.78635055652876\n", + "P-Value: 0.00046034248285934797\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.2825564971751412\n", + "T-Statistic: -4.239397969490223\n", + "P-Value: 0.024029911324908407\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1725\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1726\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -3.985191937237371\n", + "P-Value: 0.02828153983006206\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.2996468926553672\n", + "T-Statistic: -2.9888839714804116\n", + "P-Value: 0.058182452145839086\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1726\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1727\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3559426229508197\n", + "T-Statistic: -3.590780809031045\n", + "P-Value: 0.037002659109130365\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.2979284369114878\n", + "T-Statistic: -5.611049007782942\n", + "P-Value: 0.030324925803336726\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1727\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1728\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8360655737704918\n", + "Average of Other Ratios: 0.30833333333333335\n", + "T-Statistic: -25.163943419716063\n", + "P-Value: 0.00013761668634114897\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3248587570621469\n", + "T-Statistic: -6.42539604115687\n", + "P-Value: 0.0076408389421474725\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1728\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1729\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8360655737704918\n", + "Average of Other Ratios: 0.30833333333333335\n", + "T-Statistic: -14.528409507245575\n", + "P-Value: 0.0007070637377049956\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3161016949152543\n", + "T-Statistic: -4.91247301430133\n", + "P-Value: 0.016154741914472536\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1729\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1730\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.4310109289617486\n", + "T-Statistic: -3.9857409240546327\n", + "P-Value: 0.028271354653200294\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.39668079096045195\n", + "T-Statistic: -4.042742188321896\n", + "P-Value: 0.02723902806724627\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1730\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1731\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.2958333333333334\n", + "T-Statistic: -5.2910842726589395\n", + "P-Value: 0.013171332285404133\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.25748587570621473\n", + "T-Statistic: -2.8937262538718946\n", + "P-Value: 0.06282208280557401\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1731\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1732\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3568306010928962\n", + "T-Statistic: -24.813855969761136\n", + "P-Value: 0.00014350088541621058\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.30409604519774014\n", + "T-Statistic: -3.6040922833026183\n", + "P-Value: 0.036656113662429206\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1732\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1733\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.45\n", + "T-Statistic: -21.064870818149505\n", + "P-Value: 0.00023403567730180875\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.4051553672316384\n", + "T-Statistic: -6.7314755578400405\n", + "P-Value: 0.006693743268842144\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1733\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1734\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.38135245901639336\n", + "T-Statistic: -5.256718568825\n", + "P-Value: 0.013410791921047589\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.326271186440678\n", + "T-Statistic: -8.548884639029154\n", + "P-Value: 0.00336320130822139\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1734\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1735\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.39009562841530054\n", + "T-Statistic: -2.671727253283191\n", + "P-Value: 0.07557911199765509\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.2996468926553672\n", + "T-Statistic: -5.609714558498771\n", + "P-Value: 0.011196176065376142\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1735\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1736\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4108606557377049\n", + "T-Statistic: -3.0138201768645576\n", + "P-Value: 0.05703829450534057\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.37542372881355934\n", + "T-Statistic: -3.3908313960680885\n", + "P-Value: 0.04274868888333008\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1736\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1737\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.31509562841530053\n", + "T-Statistic: -12.567253014018696\n", + "P-Value: 0.0010862702962580325\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.3093220338983051\n", + "T-Statistic: -3.1999999999999997\n", + "P-Value: 0.04933184296269623\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1737\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1738\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.48333333333333334\n", + "T-Statistic: -8.472825175856576\n", + "P-Value: 0.003451637475653648\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.38201506591337103\n", + "T-Statistic: -6.299599690538863\n", + "P-Value: 0.024284334261122562\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1738\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1739\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.3583333333333333\n", + "T-Statistic: -12.770263120773526\n", + "P-Value: 0.001036015557970406\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.35021186440677965\n", + "T-Statistic: -13.369445204883407\n", + "P-Value: 0.0009045905316411595\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1739\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1740\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.3708333333333333\n", + "T-Statistic: -2.3891724073960368\n", + "P-Value: 0.09681354824211097\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.2923728813559322\n", + "T-Statistic: -5.374733592842522\n", + "P-Value: 0.012611364632413758\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1740\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1741\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.4483606557377049\n", + "T-Statistic: -12.885005111050303\n", + "P-Value: 0.0010089720709264307\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.4219632768361582\n", + "T-Statistic: -39.40000000000022\n", + "P-Value: 3.59729328035016e-05\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1741\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1742\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.40416666666666673\n", + "T-Statistic: -5.830640867335859\n", + "P-Value: 0.010049473664175738\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.3708333333333333\n", + "T-Statistic: -4.916019118203146\n", + "P-Value: 0.016122913876139212\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1742\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1743\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.37083333333333335\n", + "T-Statistic: -10.982657455064276\n", + "P-Value: 0.0016163522474045274\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3008474576271187\n", + "T-Statistic: -13.987061690606863\n", + "P-Value: 0.0007913286284483068\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1743\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1744\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.38954918032786884\n", + "T-Statistic: -4.0760180019956564\n", + "P-Value: 0.02665876696412885\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3177966101694915\n", + "T-Statistic: -4.258337907495993\n", + "P-Value: 0.023747193827929766\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1744\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1745\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.3447404371584699\n", + "T-Statistic: -3.192419176892379\n", + "P-Value: 0.0496190039928642\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.2572033898305085\n", + "T-Statistic: -4.033986241649274\n", + "P-Value: 0.027394411372010617\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1745\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1746\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.4438524590163935\n", + "T-Statistic: -1.784232222780322\n", + "P-Value: 0.17238357843904029\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3540489642184557\n", + "T-Statistic: -3.3684210526315805\n", + "P-Value: 0.07796676388459127\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1746\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1747\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.4143897996357013\n", + "T-Statistic: -8.657804801663772\n", + "P-Value: 0.013079710332824616\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.38813559322033897\n", + "T-Statistic: -4.245089541976653\n", + "P-Value: 0.02394449803344903\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1747\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1748\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.3875\n", + "T-Statistic: -5.491151239179569\n", + "P-Value: 0.011882700028816374\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3204802259887006\n", + "T-Statistic: -5.112836449747386\n", + "P-Value: 0.014477393996814832\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1748\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1749\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.46885245901639344\n", + "T-Statistic: -3.6820317950825556\n", + "P-Value: 0.03470816139558017\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.41949152542372886\n", + "T-Statistic: -6.516123792642811\n", + "P-Value: 0.007342666934462457\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1749\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1750\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.47500000000000003\n", + "T-Statistic: -3.5044017326627706\n", + "P-Value: 0.03935606956265976\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.4093220338983051\n", + "T-Statistic: -4.384615384615384\n", + "P-Value: 0.02196803265379433\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1750\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1751\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.3458333333333333\n", + "T-Statistic: -15.423254035682769\n", + "P-Value: 0.0005921188778650158\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.32902542372881355\n", + "T-Statistic: -4.153671286484361\n", + "P-Value: 0.025365339369804084\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1751\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1752\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.75\n", + "Average of Other Ratios: 0.42745901639344264\n", + "T-Statistic: -24.49428445453386\n", + "P-Value: 0.00014916835502607742\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.367090395480226\n", + "T-Statistic: -3.028199837684228\n", + "P-Value: 0.05639133971192534\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1752\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1753\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7540983606557377\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -11.683939490062333\n", + "P-Value: 0.0013469985474943292\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3800847457627119\n", + "T-Statistic: -2.6499484677906273\n", + "P-Value: 0.07699722400216412\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1753\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1754\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.4125\n", + "T-Statistic: -3.7772132331663713\n", + "P-Value: 0.03250520782748611\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3713983050847458\n", + "T-Statistic: -3.082793063741217\n", + "P-Value: 0.054017563157762766\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1754\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1755\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.3\n", + "T-Statistic: -10.255681847231036\n", + "P-Value: 0.0019765537724212886\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.2866290018832392\n", + "T-Statistic: -4.709835325623289\n", + "P-Value: 0.04224443000728903\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1755\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1756\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3360655737704918\n", + "T-Statistic: -10.610181401233849\n", + "P-Value: 0.00178889887952055\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.2532015065913371\n", + "T-Statistic: -2.2513229059561555\n", + "P-Value: 0.15321033928811528\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1756\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1757\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8360655737704918\n", + "Average of Other Ratios: 0.3875\n", + "T-Statistic: -18.026271996123526\n", + "P-Value: 0.0003723594365652728\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3763653483992467\n", + "T-Statistic: -3.2352232779643906\n", + "P-Value: 0.0837182578425499\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1757\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1758\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.3524590163934427\n", + "T-Statistic: -9.375725828304732\n", + "P-Value: 0.002570100386249655\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.33742937853107347\n", + "T-Statistic: -2.4658853737705697\n", + "P-Value: 0.09039381717094448\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1758\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1759\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.45833333333333337\n", + "T-Statistic: -7.472032792203066\n", + "P-Value: 0.004964056821106828\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.35847457627118645\n", + "T-Statistic: -5.143398239932954\n", + "P-Value: 0.014241825510816557\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1759\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1760\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.4308743169398907\n", + "T-Statistic: -4.435866726830125\n", + "P-Value: 0.04724822608025257\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3813559322033898\n", + "T-Statistic: -4.676537180435968\n", + "P-Value: 0.018469641646674063\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1760\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1761\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4208333333333334\n", + "T-Statistic: -5.724222068679529\n", + "P-Value: 0.010581602969978048\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.363135593220339\n", + "T-Statistic: -2.293068697718329\n", + "P-Value: 0.10565819843234399\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1761\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1762\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4349726775956284\n", + "T-Statistic: -3.1471966691041255\n", + "P-Value: 0.05137660775019963\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.38375706214689265\n", + "T-Statistic: -2.3710395141336553\n", + "P-Value: 0.09841141140216388\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1762\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1763\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4689207650273224\n", + "T-Statistic: -2.927463218319362\n", + "P-Value: 0.0611258808814014\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.42664783427495295\n", + "T-Statistic: -3.434253416143983\n", + "P-Value: 0.07533225481738264\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1763\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1764\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.2701502732240437\n", + "T-Statistic: -3.40514995545028\n", + "P-Value: 0.042300988302641715\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.2542372881355932\n", + "Average of Other Ratios: 0.2189265536723164\n", + "T-Statistic: -2.272727272727272\n", + "P-Value: 0.15095548648584448\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1764\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1765\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4566256830601093\n", + "T-Statistic: -3.4235368770297985\n", + "P-Value: 0.04173478390661575\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3940677966101695\n", + "T-Statistic: -1.6131196118561335\n", + "P-Value: 0.20511808054337077\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1765\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1766\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4398224043715847\n", + "T-Statistic: -10.558130916542433\n", + "P-Value: 0.0018149271537199203\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4224576271186441\n", + "T-Statistic: -2.122911802712327\n", + "P-Value: 0.1238400486795913\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1766\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1767\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.45416666666666666\n", + "T-Statistic: -5.7684452397766\n", + "P-Value: 0.010356094063164631\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3983050847457627\n", + "T-Statistic: -3.4756486205215467\n", + "P-Value: 0.04018159901970025\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1767\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1768\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.41454918032786886\n", + "T-Statistic: -2.3715832376714867\n", + "P-Value: 0.09836303411399833\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.4067796610169492\n", + "T-Statistic: -3.4985949560740326\n", + "P-Value: 0.03952103415356299\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1768\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1769\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3109289617486338\n", + "T-Statistic: -6.228218140785886\n", + "P-Value: 0.008345917876290724\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.23634651600753295\n", + "T-Statistic: -3.432432432432432\n", + "P-Value: 0.07540339116695642\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1769\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1770\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.3916666666666667\n", + "T-Statistic: -5.9956272290627375\n", + "P-Value: 0.00929176090610574\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3125\n", + "T-Statistic: -3.6292186583582198\n", + "P-Value: 0.03601320371125468\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1770\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1771\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.42745901639344264\n", + "T-Statistic: -26.610860299950033\n", + "P-Value: 0.00011643687772756474\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.38757062146892657\n", + "T-Statistic: -6.714285714285707\n", + "P-Value: 0.021470189769271106\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1771\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1772\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.35000000000000003\n", + "T-Statistic: -3.224205259523217\n", + "P-Value: 0.048428898137238975\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.30508474576271183\n", + "T-Statistic: -5.099428072506434\n", + "P-Value: 0.01458234892459302\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1772\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1773\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.3916666666666666\n", + "T-Statistic: -8.7724963613773\n", + "P-Value: 0.0031199616023164897\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.3432203389830508\n", + "T-Statistic: -4.943805429141822\n", + "P-Value: 0.015876322370823124\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1773\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1774\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.33128415300546443\n", + "T-Statistic: -3.6830493742159134\n", + "P-Value: 0.03468361534744764\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.28255649717514125\n", + "T-Statistic: -7.881999450070924\n", + "P-Value: 0.004255505421232205\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1774\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1775\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.4125\n", + "T-Statistic: -4.161173917605138\n", + "P-Value: 0.025244699113682725\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.40494350282485875\n", + "T-Statistic: -5.023746418203985\n", + "P-Value: 0.015193736814334122\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1775\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1776\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.49583333333333335\n", + "T-Statistic: -2.859986118208274\n", + "P-Value: 0.0645777020694818\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.4595338983050848\n", + "T-Statistic: -2.392754433686525\n", + "P-Value: 0.09650164759323027\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1776\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1777\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.435724043715847\n", + "T-Statistic: -3.1832159126292936\n", + "P-Value: 0.04997046721617775\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.35459039548022603\n", + "T-Statistic: -5.888771927947271\n", + "P-Value: 0.009773561677083785\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1777\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1778\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.4166666666666667\n", + "T-Statistic: -8.499595555559111\n", + "P-Value: 0.003420163998746116\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.32478813559322034\n", + "T-Statistic: -12.454354190804333\n", + "P-Value: 0.0011156250386489563\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1778\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1779\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.46038251366120214\n", + "T-Statistic: -2.3871021119027693\n", + "P-Value: 0.09699437660783286\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3755649717514124\n", + "T-Statistic: -3.3684353047674622\n", + "P-Value: 0.04346109463220811\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1779\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1780\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3898224043715847\n", + "T-Statistic: -1.856072621295239\n", + "P-Value: 0.16045407134930564\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3672316384180791\n", + "T-Statistic: -4.082482904638627\n", + "P-Value: 0.026547885467199526\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1780\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1781\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.3833333333333333\n", + "T-Statistic: -3.203403812225747\n", + "P-Value: 0.049203590003784436\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.35402542372881357\n", + "T-Statistic: -2.5536843901385566\n", + "P-Value: 0.08367347896165804\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1781\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1782\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4000000000000001\n", + "T-Statistic: -37.585613102377934\n", + "P-Value: 4.1428594619016965e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4175141242937853\n", + "T-Statistic: -1.946993604286342\n", + "P-Value: 0.1467043534415175\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1782\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1783\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.3625\n", + "T-Statistic: -9.790972522123287\n", + "P-Value: 0.002264226155807123\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.37966101694915255\n", + "T-Statistic: -3.993713653195303\n", + "P-Value: 0.028123969733242882\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1783\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1784\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.3983606557377049\n", + "T-Statistic: -5.666139775569049\n", + "P-Value: 0.010887658149858946\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.354590395480226\n", + "T-Statistic: -2.8307905599569243\n", + "P-Value: 0.06614653353035789\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1784\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1785\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.37698087431693994\n", + "T-Statistic: -3.066930962984633\n", + "P-Value: 0.054694091557813726\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.32090395480225986\n", + "T-Statistic: -3.177686792530804\n", + "P-Value: 0.05018313128134387\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1785\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1786\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.4291666666666667\n", + "T-Statistic: -3.794152560630122\n", + "P-Value: 0.03213210729307197\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.4304378531073446\n", + "T-Statistic: -6.772008114046649\n", + "P-Value: 0.006580141599024013\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1786\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1787\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.4375\n", + "T-Statistic: -4.005635792635015\n", + "P-Value: 0.027905415031102557\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3839689265536724\n", + "T-Statistic: -5.701874626813004\n", + "P-Value: 0.010698008944756929\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1787\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1788\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4398907103825137\n", + "T-Statistic: -15.578682000811366\n", + "P-Value: 0.0005747431214231546\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.3542372881355932\n", + "T-Statistic: -5.854047900359961\n", + "P-Value: 0.009937162378535997\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1788\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1789\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4480191256830601\n", + "T-Statistic: -6.166892456957159\n", + "P-Value: 0.008582542762464446\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3412429378531074\n", + "T-Statistic: -3.8779880896871726\n", + "P-Value: 0.030364316576825364\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1789\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1790\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.44760928961748636\n", + "T-Statistic: -3.4952014484831047\n", + "P-Value: 0.039617848196352085\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.42196327683615825\n", + "T-Statistic: -3.458948005905017\n", + "P-Value: 0.04067122554425211\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1790\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1791\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.3541666666666667\n", + "T-Statistic: -3.9615442295558925\n", + "P-Value: 0.028724785655088337\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3135593220338983\n", + "T-Statistic: -1.8272855362744203\n", + "P-Value: 0.16511600726806364\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1791\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1792\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8032786885245902\n", + "Average of Other Ratios: 0.29583333333333334\n", + "T-Statistic: -12.539145243620842\n", + "P-Value: 0.0010934817370295948\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3377118644067797\n", + "T-Statistic: -6.269358747755155\n", + "P-Value: 0.00819196116993917\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1792\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1793\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.45416666666666666\n", + "T-Statistic: -7.1071028723917875\n", + "P-Value: 0.0057315850409424066\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5932203389830508\n", + "Average of Other Ratios: 0.40939265536723163\n", + "T-Statistic: -13.436639703627074\n", + "P-Value: 0.0008912634414880094\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1793\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1794\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.42083333333333334\n", + "T-Statistic: -6.840395759439469\n", + "P-Value: 0.006394156517694729\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.2910310734463277\n", + "T-Statistic: -4.146372958429369\n", + "P-Value: 0.025483409105270218\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1794\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1795\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4083333333333334\n", + "T-Statistic: -4.492763882679988\n", + "P-Value: 0.020579223849711902\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.3313559322033898\n", + "T-Statistic: -1.9155635193164227\n", + "P-Value: 0.19549520636629938\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1795\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1796\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.41905737704918034\n", + "T-Statistic: -6.292690992312831\n", + "P-Value: 0.008106302500641403\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.38728813559322034\n", + "T-Statistic: -2.7837837837837838\n", + "P-Value: 0.10845063612547547\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1796\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1797\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.4064890710382514\n", + "T-Statistic: -7.456642804764441\n", + "P-Value: 0.004993590238372753\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.321045197740113\n", + "T-Statistic: -5.483588555093928\n", + "P-Value: 0.011928337117442485\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1797\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1798\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.4083333333333334\n", + "T-Statistic: -2.8639314065795065\n", + "P-Value: 0.06436927783811348\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3612994350282486\n", + "T-Statistic: -14.789473684210558\n", + "P-Value: 0.042980044870241256\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1798\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1799\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.4416666666666667\n", + "T-Statistic: -4.673530022214723\n", + "P-Value: 0.018501844031635108\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3962570621468926\n", + "T-Statistic: -3.830072253349244\n", + "P-Value: 0.0313589410287926\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1799\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1800\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.3916666666666667\n", + "T-Statistic: -4.31785074614014\n", + "P-Value: 0.02288635431540596\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3246468926553672\n", + "T-Statistic: -3.659896369736786\n", + "P-Value: 0.035247654519723955\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1800\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1801\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.3125\n", + "T-Statistic: -21.98360655737705\n", + "P-Value: 0.0002060384328133572\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.2919962335216572\n", + "T-Statistic: -3.5407885315000662\n", + "P-Value: 0.07133333882574665\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1801\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1802\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.38333333333333336\n", + "T-Statistic: -4.785407410045699\n", + "P-Value: 0.017351257215718095\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3584039548022599\n", + "T-Statistic: -3.1685329501250483\n", + "P-Value: 0.050537730061605154\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1802\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1803\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.35\n", + "T-Statistic: -3.0834492425511746\n", + "P-Value: 0.05398980423292988\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.32076271186440675\n", + "T-Statistic: -6.808401479691212\n", + "P-Value: 0.006480293784839257\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1803\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1804\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.39166666666666666\n", + "T-Statistic: -3.7805287666533993\n", + "P-Value: 0.032431745470329315\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.34625706214689267\n", + "T-Statistic: -2.0407541348149305\n", + "P-Value: 0.13394942774558222\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1804\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1805\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.39166666666666666\n", + "T-Statistic: -5.539911623921449\n", + "P-Value: 0.011593870922493347\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3516949152542373\n", + "T-Statistic: -5.158515399269249\n", + "P-Value: 0.014127150588176329\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1805\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1806\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.410724043715847\n", + "T-Statistic: -11.448148657407378\n", + "P-Value: 0.0014304164055433837\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.37146892655367225\n", + "T-Statistic: -5.367066741284801\n", + "P-Value: 0.012661377799601295\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1806\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1807\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.4291666666666667\n", + "T-Statistic: -3.064527056875999\n", + "P-Value: 0.05479754804131916\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3967514124293785\n", + "T-Statistic: -2.4964946840644773\n", + "P-Value: 0.08797820274589993\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1807\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1808\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4691256830601093\n", + "T-Statistic: -4.840779946644281\n", + "P-Value: 0.016816135457307743\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4745762711864407\n", + "T-Statistic: -2.3551362310207575\n", + "P-Value: 0.09983927564004276\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1808\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1809\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.45669398907103825\n", + "T-Statistic: -4.048874395684384\n", + "P-Value: 0.02713088105493987\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4009180790960452\n", + "T-Statistic: -8.122451922976826\n", + "P-Value: 0.003901259023001135\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1809\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1810\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.39583333333333337\n", + "T-Statistic: -2.771678715670956\n", + "P-Value: 0.0694710760166772\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3456920903954802\n", + "T-Statistic: -4.094924639556887\n", + "P-Value: 0.02633616071520719\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1810\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1811\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.44836065573770495\n", + "T-Statistic: -3.137402547907732\n", + "P-Value: 0.051767564979331604\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3463276836158192\n", + "T-Statistic: -3.5851541120965997\n", + "P-Value: 0.03715039979658868\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1811\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1812\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -9.207415437764503\n", + "P-Value: 0.0027096685823184735\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3125\n", + "T-Statistic: -2.848001248439178\n", + "P-Value: 0.06521603391989962\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1812\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1813\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -6.7690431384831955\n", + "P-Value: 0.006588365191149356\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.2955508474576271\n", + "T-Statistic: -5.314610961015995\n", + "P-Value: 0.013010610801463965\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1813\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1814\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.43995901639344265\n", + "T-Statistic: -6.955531695206198\n", + "P-Value: 0.0060963729996066736\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3711864406779661\n", + "T-Statistic: -6.051234040593998\n", + "P-Value: 0.009053328678904286\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1814\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1815\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -4.17397260708643\n", + "P-Value: 0.02504060302241229\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.2994350282485876\n", + "T-Statistic: -2.6261286571944504\n", + "P-Value: 0.07858610153137385\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1815\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1816\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.3875\n", + "T-Statistic: -13.414708580029796\n", + "P-Value: 0.0008955843183091936\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.35593220338983056\n", + "T-Statistic: -3.925344995453686\n", + "P-Value: 0.02942075287920854\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1816\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1817\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.5266393442622951\n", + "T-Statistic: -2.0100838668302545\n", + "P-Value: 0.13797226511673932\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.43481638418079094\n", + "T-Statistic: -4.495559790237215\n", + "P-Value: 0.02054484962621015\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1817\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1818\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3726775956284153\n", + "T-Statistic: -3.954974925804998\n", + "P-Value: 0.028849502427145693\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3416666666666667\n", + "T-Statistic: -5.211728279194447\n", + "P-Value: 0.01373294914761014\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1818\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1819\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.44583333333333336\n", + "T-Statistic: -6.313455744839042\n", + "P-Value: 0.00803105673664604\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4387711864406779\n", + "T-Statistic: -3.071254985149466\n", + "P-Value: 0.05450861721232613\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1819\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1820\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.44815573770491807\n", + "T-Statistic: -4.067460467766922\n", + "P-Value: 0.02680645933365517\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3923022598870056\n", + "T-Statistic: -2.724050147464563\n", + "P-Value: 0.07230209895638379\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1820\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1821\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.475\n", + "T-Statistic: -3.0602917588065157\n", + "P-Value: 0.054980421525545886\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.576271186440678\n", + "Average of Other Ratios: 0.40494350282485875\n", + "T-Statistic: -6.15102387874345\n", + "P-Value: 0.008645202687168257\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1821\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1822\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3775273224043716\n", + "T-Statistic: -9.402775448335925\n", + "P-Value: 0.0025485643327779633\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3374293785310734\n", + "T-Statistic: -2.6487965059751333\n", + "P-Value: 0.07707314575519472\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1822\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1823\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.36489071038251364\n", + "T-Statistic: -8.409157084603672\n", + "P-Value: 0.003528044949249387\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.2919962335216572\n", + "T-Statistic: -9.926980271680543\n", + "P-Value: 0.009995759216947474\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1823\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1824\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.42725409836065575\n", + "T-Statistic: -3.221457895636802\n", + "P-Value: 0.04853032779203252\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3204802259887006\n", + "T-Statistic: -6.442102133047244\n", + "P-Value: 0.007584755136376526\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1824\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1825\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.4625\n", + "T-Statistic: -3.0100455180506156\n", + "P-Value: 0.05720966272875764\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.4279661016949153\n", + "T-Statistic: -4.9770903720375195\n", + "P-Value: 0.015587358121496146\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1825\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1826\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.385792349726776\n", + "T-Statistic: -4.6686976061393075\n", + "P-Value: 0.018553745232426106\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3432203389830508\n", + "T-Statistic: -10.881765043296808\n", + "P-Value: 0.0016608270081661155\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1826\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1827\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.37732240437158465\n", + "T-Statistic: -5.919359446594679\n", + "P-Value: 0.009632363464382891\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.38799435028248586\n", + "T-Statistic: -1.7579248960751477\n", + "P-Value: 0.17700539897601802\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1827\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1828\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.45\n", + "T-Statistic: -2.3790961945021833\n", + "P-Value: 0.0976975284567096\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.36271186440677966\n", + "T-Statistic: -3.4834542887072035\n", + "P-Value: 0.03995532706040686\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1828\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1829\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.47274590163934427\n", + "T-Statistic: -6.685134180648596\n", + "P-Value: 0.006826809830237584\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.40932203389830507\n", + "T-Statistic: -12.89316271749542\n", + "P-Value: 0.0010070852904372337\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1829\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1830\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.42500000000000004\n", + "T-Statistic: -1.7723627686294894\n", + "P-Value: 0.17445150254614536\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3586158192090395\n", + "T-Statistic: -5.301653087443131\n", + "P-Value: 0.013098812973106142\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1830\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1831\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4687158469945355\n", + "T-Statistic: -2.42952422971014\n", + "P-Value: 0.09336973169699041\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.41101694915254233\n", + "T-Statistic: -2.6934842013379563\n", + "P-Value: 0.07419455966723891\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1831\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1832\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.44781420765027324\n", + "T-Statistic: -2.175312723682659\n", + "P-Value: 0.1178663975785963\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.36694915254237287\n", + "T-Statistic: -2.416081794014973\n", + "P-Value: 0.0945001250957143\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1832\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1833\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.42698087431693993\n", + "T-Statistic: -2.926838266377036\n", + "P-Value: 0.06115677716147604\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.35028248587570626\n", + "T-Statistic: -4.358898943540671\n", + "P-Value: 0.02231600572520053\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1833\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1834\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4226775956284153\n", + "T-Statistic: -3.0320096719785434\n", + "P-Value: 0.05622147844106881\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.35882768361581924\n", + "T-Statistic: -2.596202410283425\n", + "P-Value: 0.08064011223189176\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1834\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1835\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.42500000000000004\n", + "T-Statistic: -8.165010792168419\n", + "P-Value: 0.0038426628271698176\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3968220338983051\n", + "T-Statistic: -2.5012480459007254\n", + "P-Value: 0.08761020945997577\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1835\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1836\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.40689890710382515\n", + "T-Statistic: -13.281769925732059\n", + "P-Value: 0.00092238087463812\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.40918079096045196\n", + "T-Statistic: -3.6547443425101043\n", + "P-Value: 0.03537475739142397\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1836\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1837\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7540983606557377\n", + "Average of Other Ratios: 0.4541666666666667\n", + "T-Statistic: -7.998178506375221\n", + "P-Value: 0.004079263296996052\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.4093220338983051\n", + "T-Statistic: -6.728880964155069\n", + "P-Value: 0.006701102825924804\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1837\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1838\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -9.166095697580067\n", + "P-Value: 0.002745459142872884\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3880649717514124\n", + "T-Statistic: -3.6962350907584107\n", + "P-Value: 0.03436754879976233\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1838\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1839\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3903688524590164\n", + "T-Statistic: -2.6475959716199546\n", + "P-Value: 0.07715236726431632\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3135593220338983\n", + "T-Statistic: -3.0363270659271957\n", + "P-Value: 0.056029764059456005\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1839\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1840\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4352459016393443\n", + "T-Statistic: -2.217999452996241\n", + "P-Value: 0.11325398002800678\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.3624293785310735\n", + "T-Statistic: -4.007620942190811\n", + "P-Value: 0.02786923613375584\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1840\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1841\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -2.7939211613370594\n", + "P-Value: 0.06819623625179277\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3442090395480226\n", + "T-Statistic: -1.1660649819494582\n", + "P-Value: 0.4512877748622783\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1841\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1842\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3810109289617486\n", + "T-Statistic: -3.117346730699978\n", + "P-Value: 0.052579921647726034\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.36299435028248583\n", + "T-Statistic: -5.564047150200577\n", + "P-Value: 0.011454300879796516\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1842\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1843\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.39419398907103825\n", + "T-Statistic: -3.6055517374511754\n", + "P-Value: 0.036618371391318225\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.35593220338983045\n", + "T-Statistic: -4.53155602414888\n", + "P-Value: 0.020108869526374775\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1843\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1844\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.41250000000000003\n", + "T-Statistic: -3.8013214837489318\n", + "P-Value: 0.03197586070428311\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3728813559322034\n", + "T-Statistic: -3.2995600879804483\n", + "P-Value: 0.04574853906846063\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1844\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1845\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.41666666666666663\n", + "T-Statistic: -5.316864560968578\n", + "P-Value: 0.01299535017353051\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3801553672316384\n", + "T-Statistic: -2.6218256817858103\n", + "P-Value: 0.0788774316850411\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1845\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1846\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -11.831027585036374\n", + "P-Value: 0.001298210003952447\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.34625706214689267\n", + "T-Statistic: -2.2952786442989725\n", + "P-Value: 0.10544407020716878\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1846\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1847\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -8.063231850117097\n", + "P-Value: 0.003984775525196651\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.34194915254237285\n", + "T-Statistic: -7.414573731136393\n", + "P-Value: 0.005075512686812491\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1847\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1848\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3689890710382514\n", + "T-Statistic: -4.439726457213278\n", + "P-Value: 0.021245642150683707\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.29540960451977405\n", + "T-Statistic: -4.994116161502297\n", + "P-Value: 0.015442194581693753\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1848\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1849\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.3025273224043716\n", + "T-Statistic: -11.897761281046622\n", + "P-Value: 0.001276847670564472\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.2660310734463277\n", + "T-Statistic: -4.308319914295902\n", + "P-Value: 0.02302147063382807\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1849\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1850\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.44166666666666665\n", + "T-Statistic: -4.990732440102222\n", + "P-Value: 0.015470903850084424\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.392090395480226\n", + "T-Statistic: -3.99713035331454\n", + "P-Value: 0.028061110834153927\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1850\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1851\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3980874316939891\n", + "T-Statistic: -3.5674165199732903\n", + "P-Value: 0.03762109314075504\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.34745762711864403\n", + "T-Statistic: -3.430686205868627\n", + "P-Value: 0.04151723186686211\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1851\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1852\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.40273224043715844\n", + "T-Statistic: -4.589204955202506\n", + "P-Value: 0.019435305052459464\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.35854519774011295\n", + "T-Statistic: -2.331934255279095\n", + "P-Value: 0.10196787590396043\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1852\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1853\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3774590163934426\n", + "T-Statistic: -13.772608526452023\n", + "P-Value: 0.0008284008874866314\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.2998587570621469\n", + "T-Statistic: -5.194000650735758\n", + "P-Value: 0.013862662574053051\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1853\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1854\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.39405737704918037\n", + "T-Statistic: -4.944621149115616\n", + "P-Value: 0.015869157656486683\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.2990819209039548\n", + "T-Statistic: -3.51802668500397\n", + "P-Value: 0.038972429516855533\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1854\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1855\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.37315573770491806\n", + "T-Statistic: -4.038348082595869\n", + "P-Value: 0.027316863729051996\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.34173728813559323\n", + "T-Statistic: -2.5913248074535873\n", + "P-Value: 0.08098113762773655\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1855\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1856\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.40259562841530055\n", + "T-Statistic: -3.6156991795445577\n", + "P-Value: 0.0363573206604073\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3347457627118644\n", + "T-Statistic: -5.6429955425120175\n", + "P-Value: 0.011012841275964533\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1856\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1857\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -10.42437348156729\n", + "P-Value: 0.0018841578114493912\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3794491525423729\n", + "T-Statistic: -3.26894241254243\n", + "P-Value: 0.04681436566609524\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1857\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1858\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4603142076502732\n", + "T-Statistic: -2.0946414252764485\n", + "P-Value: 0.1272128293571138\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.40084745762711865\n", + "T-Statistic: -3.985615868631576\n", + "P-Value: 0.028273674351328677\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1858\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1859\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.41905737704918034\n", + "T-Statistic: -3.9625676262993768\n", + "P-Value: 0.02870541916232176\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.34597457627118644\n", + "T-Statistic: -6.54645011829731\n", + "P-Value: 0.007246419253764158\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1859\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1860\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.29166666666666663\n", + "T-Statistic: -14.478449297651643\n", + "P-Value: 0.0007143254209499869\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.2955508474576271\n", + "T-Statistic: -3.1245537578699243\n", + "P-Value: 0.05228616491106145\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1860\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1861\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.38749999999999996\n", + "T-Statistic: -10.080682221460885\n", + "P-Value: 0.0020788624411748344\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.2823446327683616\n", + "T-Statistic: -3.3366481297841037\n", + "P-Value: 0.04449831598300532\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1861\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1862\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.37916666666666665\n", + "T-Statistic: -13.36742611721058\n", + "P-Value: 0.0009049950751945303\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3898305084745763\n", + "T-Statistic: -4.966017854713506\n", + "P-Value: 0.015682718436889013\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1862\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1863\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.45\n", + "T-Statistic: -2.0479340472449516\n", + "P-Value: 0.13302782647013972\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3248587570621469\n", + "T-Statistic: -3.666081619407081\n", + "P-Value: 0.03509583190764937\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1863\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1864\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.42916666666666664\n", + "T-Statistic: -4.986422813713202\n", + "P-Value: 0.015507569745157936\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.35466101694915253\n", + "T-Statistic: -3.9441648534374134\n", + "P-Value: 0.029056249071256115\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1864\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1865\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.441712204007286\n", + "T-Statistic: -2.327272727272727\n", + "P-Value: 0.14541235094098032\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.36864406779661013\n", + "T-Statistic: -1.786828180527595\n", + "P-Value: 0.17193506795662192\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1865\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1866\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.45997267759562843\n", + "T-Statistic: -3.7679579839771797\n", + "P-Value: 0.03271140841418524\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6440677966101694\n", + "Average of Other Ratios: 0.4304378531073446\n", + "T-Statistic: -11.862745098039216\n", + "P-Value: 0.001287998117223525\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1866\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1867\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4976775956284153\n", + "T-Statistic: -5.300050196391964\n", + "P-Value: 0.013109777815587411\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.576271186440678\n", + "Average of Other Ratios: 0.43036723163841806\n", + "T-Statistic: -3.707083667393408\n", + "P-Value: 0.034110263406932406\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1867\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1868\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8032786885245902\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -31.928961748633885\n", + "P-Value: 6.751260437384004e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.379590395480226\n", + "T-Statistic: -3.550388805186171\n", + "P-Value: 0.03808013155197999\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1868\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1869\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3358606557377049\n", + "T-Statistic: -2.42291804775351\n", + "P-Value: 0.09392318031625656\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.2838983050847458\n", + "T-Statistic: -5.199999999999993\n", + "P-Value: 0.013818586905132142\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1869\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1870\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.4194672131147541\n", + "T-Statistic: -3.4472491733646837\n", + "P-Value: 0.041018740078086716\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3840395480225988\n", + "T-Statistic: -3.832436694700159\n", + "P-Value: 0.03130888812484261\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1870\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1871\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.39583333333333337\n", + "T-Statistic: -4.526712258059946\n", + "P-Value: 0.02016683338787022\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3586864406779661\n", + "T-Statistic: -3.3503877475316646\n", + "P-Value: 0.04404617677686765\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1871\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1872\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.48101092896174863\n", + "T-Statistic: -3.9726503315641613\n", + "P-Value: 0.02851551151540454\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.39244350282485874\n", + "T-Statistic: -9.21746279562317\n", + "P-Value: 0.0027010591887466347\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1872\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1873\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.4441256830601093\n", + "T-Statistic: -9.737517564732535\n", + "P-Value: 0.00230079619793927\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3964689265536723\n", + "T-Statistic: -2.3349757965803564\n", + "P-Value: 0.10168572245232489\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1873\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1874\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7704918032786885\n", + "Average of Other Ratios: 0.38749999999999996\n", + "T-Statistic: -6.709762545561513\n", + "P-Value: 0.006755661879342611\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.40911016949152545\n", + "T-Statistic: -2.509542503119258\n", + "P-Value: 0.08697258350325465\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1874\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1875\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.3202185792349727\n", + "T-Statistic: -8.067567567567563\n", + "P-Value: 0.015019105066650768\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3287429378531073\n", + "T-Statistic: -2.9086605723710677\n", + "P-Value: 0.06206405682692753\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1875\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1876\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4229508196721311\n", + "T-Statistic: -4.277954683338313\n", + "P-Value: 0.023458887518037853\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3425141242937853\n", + "T-Statistic: -2.187920293054828\n", + "P-Value: 0.11648102879543593\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1876\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1877\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.4062841530054645\n", + "T-Statistic: -2.608245612118408\n", + "P-Value: 0.0798056461655953\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.36694915254237287\n", + "T-Statistic: -2.2961086998148015\n", + "P-Value: 0.10536377934120532\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1877\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1878\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.423155737704918\n", + "T-Statistic: -3.308126758133395\n", + "P-Value: 0.045455844114838176\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.4152542372881356\n", + "T-Statistic: -3.175846343890085\n", + "P-Value: 0.05025417308948302\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1878\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1879\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.8032786885245902\n", + "Average of Other Ratios: 0.2916666666666667\n", + "T-Statistic: -13.189427229234468\n", + "P-Value: 0.0009416245667257401\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3559322033898305\n", + "Average of Other Ratios: 0.29971751412429376\n", + "T-Statistic: -3.099929158257806\n", + "P-Value: 0.05329849219831802\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1879\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1880\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.40239071038251367\n", + "T-Statistic: -2.8278186890378114\n", + "P-Value: 0.06630888074205549\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3305084745762712\n", + "T-Statistic: -6.072629199681584\n", + "P-Value: 0.008963729862100802\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1880\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1881\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4270491803278688\n", + "T-Statistic: -2.1944780188324806\n", + "P-Value: 0.11576816528411227\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.31101694915254235\n", + "T-Statistic: -7.153846153846161\n", + "P-Value: 0.08841694696574594\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1881\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1882\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.3773224043715847\n", + "T-Statistic: -3.575689868461351\n", + "P-Value: 0.03740060612655586\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3038135593220339\n", + "T-Statistic: -7.40913346851445\n", + "P-Value: 0.005086235812941151\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1882\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1883\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.43333333333333335\n", + "T-Statistic: -6.573390283759843\n", + "P-Value: 0.007162305426399894\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.4152542372881356\n", + "T-Statistic: -2.2929844695893116\n", + "P-Value: 0.10566636994498525\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1883\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1884\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.425\n", + "T-Statistic: -7.134072833363263\n", + "P-Value: 0.005669714559524623\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.37521186440677967\n", + "T-Statistic: -4.801612169541382\n", + "P-Value: 0.017192389686773397\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1884\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1885\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.39999999999999997\n", + "T-Statistic: -5.171937592179927\n", + "P-Value: 0.014026340881299086\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.33785310734463275\n", + "T-Statistic: -3.9583928774422366\n", + "P-Value: 0.028784526660768115\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1885\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1886\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4687158469945355\n", + "T-Statistic: -2.666321218691447\n", + "P-Value: 0.07592808657201537\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3879237288135593\n", + "T-Statistic: -2.7149672381577274\n", + "P-Value: 0.07285814285484306\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1886\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1887\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4603142076502732\n", + "T-Statistic: -3.6716000925111048\n", + "P-Value: 0.03496108120386053\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.38983050847457623\n", + "T-Statistic: -7.2205339907178026\n", + "P-Value: 0.005477206760423499\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1887\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1888\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.41045081967213115\n", + "T-Statistic: -1.9837288411299292\n", + "P-Value: 0.14154345106445665\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3686440677966102\n", + "T-Statistic: -5.918640302493723\n", + "P-Value: 0.009635652367993155\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1888\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1889\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4400273224043716\n", + "T-Statistic: -4.325688830675532\n", + "P-Value: 0.022776003224226602\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3987758945386064\n", + "T-Statistic: -1.8820851842053508\n", + "P-Value: 0.20054056943775883\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1889\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1890\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.42288251366120216\n", + "T-Statistic: -2.3707547764032237\n", + "P-Value: 0.09843675721882074\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.43015536723163844\n", + "T-Statistic: -2.973639372537061\n", + "P-Value: 0.05889612816583925\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1890\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1891\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.42500000000000004\n", + "T-Statistic: -3.0224716644609666\n", + "P-Value: 0.056647945467235325\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.33361581920903954\n", + "T-Statistic: -4.658888265058127\n", + "P-Value: 0.018659682503791214\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1891\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1892\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.3791666666666667\n", + "T-Statistic: -5.283658664721413\n", + "P-Value: 0.013222597948006223\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.37097457627118646\n", + "T-Statistic: -3.2449363634867825\n", + "P-Value: 0.04767210916456234\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1892\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1893\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.36885245901639346\n", + "T-Statistic: -2.7563407387163426\n", + "P-Value: 0.07036747746122053\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.36016949152542377\n", + "T-Statistic: -2.612964350541548\n", + "P-Value: 0.07948159398225181\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1893\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1894\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.32499999999999996\n", + "T-Statistic: -16.552311734273484\n", + "P-Value: 0.0004799737975285094\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.32937853107344633\n", + "T-Statistic: -4.435658555321488\n", + "P-Value: 0.021297902760540714\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1894\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1895\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.43128415300546447\n", + "T-Statistic: -5.492433465454693\n", + "P-Value: 0.011874985075137982\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3543785310734463\n", + "T-Statistic: -7.723907549595242\n", + "P-Value: 0.004511849154258445\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1895\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1896\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.4478142076502733\n", + "T-Statistic: -4.942348751891569\n", + "P-Value: 0.01588912728706134\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.41765536723163843\n", + "T-Statistic: -6.264933372064462\n", + "P-Value: 0.008208341834463534\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1896\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1897\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.40225409836065573\n", + "T-Statistic: -6.568502779421056\n", + "P-Value: 0.007177469697378324\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3501412429378531\n", + "T-Statistic: -4.144463927530725\n", + "P-Value: 0.02551440964648421\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1897\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1898\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3775273224043716\n", + "T-Statistic: -10.30131962224801\n", + "P-Value: 0.0019509748009049809\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.30798022598870056\n", + "T-Statistic: -8.186715143005392\n", + "P-Value: 0.003813227168001511\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1898\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1899\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4235655737704918\n", + "T-Statistic: -2.574541708593826\n", + "P-Value: 0.08216821527460087\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3123587570621469\n", + "T-Statistic: -4.795397246883075\n", + "P-Value: 0.017253094776815005\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1899\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1900\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.45614754098360655\n", + "T-Statistic: -2.6559601893971574\n", + "P-Value: 0.07660250967292412\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.40254237288135597\n", + "T-Statistic: -3.012278225167804\n", + "P-Value: 0.0571082206033358\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1900\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1901\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.32916666666666666\n", + "T-Statistic: -11.674705169820832\n", + "P-Value: 0.0013501421719868342\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3008474576271186\n", + "T-Statistic: -3.099405715532204\n", + "P-Value: 0.053320277974167773\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1901\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1902\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4108606557377049\n", + "T-Statistic: -6.418429391739877\n", + "P-Value: 0.007664387655280491\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.29562146892655367\n", + "T-Statistic: -3.85816763155035\n", + "P-Value: 0.030770761179515826\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1902\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1903\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7540983606557377\n", + "Average of Other Ratios: 0.4208333333333334\n", + "T-Statistic: -15.206265335945545\n", + "P-Value: 0.0006175648734416628\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3714689265536723\n", + "T-Statistic: -4.782389009187055\n", + "P-Value: 0.017381059516950674\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1903\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1904\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.3625\n", + "T-Statistic: -3.600412240200614\n", + "P-Value: 0.036751502378933966\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3220338983050848\n", + "T-Statistic: -3.7931280643843954\n", + "P-Value: 0.03215451606094398\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1904\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1905\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7\n", + "Average of Other Ratios: 0.419603825136612\n", + "T-Statistic: -5.8545065009896256\n", + "P-Value: 0.00993497840099364\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.40084745762711865\n", + "T-Statistic: -2.4921620337819483\n", + "P-Value: 0.08831527775786965\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1905\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1906\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.4602459016393443\n", + "T-Statistic: -5.345769495574332\n", + "P-Value: 0.012801672818638945\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.4009180790960452\n", + "T-Statistic: -9.991472499692488\n", + "P-Value: 0.0021337257653321886\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1906\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1907\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.3818989071038251\n", + "T-Statistic: -15.322633755583363\n", + "P-Value: 0.0006037415969847308\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.36666666666666664\n", + "Average of Other Ratios: 0.3220338983050848\n", + "T-Statistic: -4.561067126598043\n", + "P-Value: 0.019760347058840888\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1907\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1908\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.48750000000000004\n", + "T-Statistic: -3.044989012722217\n", + "P-Value: 0.055647600875460466\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4262005649717514\n", + "T-Statistic: -6.726801374615161\n", + "P-Value: 0.006707009265801296\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1908\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1909\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.45\n", + "T-Statistic: -3.5859661472271527\n", + "P-Value: 0.03712903167578981\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.34611581920903955\n", + "T-Statistic: -7.571218430433462\n", + "P-Value: 0.004779142305515517\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1909\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1910\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.34822404371584703\n", + "T-Statistic: -4.589291871691855\n", + "P-Value: 0.019434311800169517\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.286864406779661\n", + "T-Statistic: -11.301306940901064\n", + "P-Value: 0.0014858566277749805\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1910\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1911\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -3.1680031770414296\n", + "P-Value: 0.050558348943150866\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.32076271186440675\n", + "T-Statistic: -8.564496343017598\n", + "P-Value: 0.0033454209602583485\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1911\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1912\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.3916666666666666\n", + "T-Statistic: -13.752294116926654\n", + "P-Value: 0.0008320313667330868\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3628531073446328\n", + "T-Statistic: -4.591316366529474\n", + "P-Value: 0.019411195178700307\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1912\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1913\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.36038251366120216\n", + "T-Statistic: -4.510639994744203\n", + "P-Value: 0.02036072397240279\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3208333333333333\n", + "T-Statistic: -10.424077229245048\n", + "P-Value: 0.0018843150010148248\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1913\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1914\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4918032786885246\n", + "Average of Other Ratios: 0.36666666666666664\n", + "T-Statistic: -2.214046423571898\n", + "P-Value: 0.1136718934082652\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.30508474576271183\n", + "T-Statistic: -3.4430305632479454\n", + "P-Value: 0.04114498062149175\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1914\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1915\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.47540983606557374\n", + "Average of Other Ratios: 0.44166666666666665\n", + "T-Statistic: -2.3377953522924186\n", + "P-Value: 0.10142500753372925\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.36709039548022593\n", + "T-Statistic: -9.92864680939292\n", + "P-Value: 0.0021735197350935946\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1915\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1916\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.3445355191256831\n", + "T-Statistic: -8.421776890073554\n", + "P-Value: 0.0035127236241960158\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.3305084745762712\n", + "T-Statistic: -41.4537493278151\n", + "P-Value: 3.0893720674523154e-05\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1916\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1917\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.43196721311475406\n", + "T-Statistic: -3.1667548890956354\n", + "P-Value: 0.05060697458442003\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.33785310734463275\n", + "T-Statistic: -7.8732869320554375\n", + "P-Value: 0.004269124399355101\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1917\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1918\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4644808743169399\n", + "T-Statistic: -2.238658218669735\n", + "P-Value: 0.11109977799644323\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5932203389830508\n", + "Average of Other Ratios: 0.38834745762711864\n", + "T-Statistic: -10.638370665053376\n", + "P-Value: 0.0017750090525035277\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1918\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1919\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3859289617486339\n", + "T-Statistic: -4.12442499010612\n", + "P-Value: 0.025842773831512598\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3389830508474576\n", + "T-Statistic: -3.9436024140371986\n", + "P-Value: 0.029067057994347754\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1919\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1920\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3979508196721311\n", + "T-Statistic: -6.410996076998129\n", + "P-Value: 0.00768961910808135\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3209039548022599\n", + "T-Statistic: -5.521881967348604\n", + "P-Value: 0.011699587084326106\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1920\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1921\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.38579234972677595\n", + "T-Statistic: -3.247114142040317\n", + "P-Value: 0.04759348039328008\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.29964689265536726\n", + "T-Statistic: -5.367242638173358\n", + "P-Value: 0.01266022746944358\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1921\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1922\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7868852459016393\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -32.828301710168866\n", + "P-Value: 6.212646695041301e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3714689265536723\n", + "T-Statistic: -5.21158171207224\n", + "P-Value: 0.013734015073651634\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1922\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1923\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -9.652812843530201\n", + "P-Value: 0.002360364565844294\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.35035310734463276\n", + "T-Statistic: -4.9456020241512695\n", + "P-Value: 0.015860547909964277\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1923\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1924\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.33749999999999997\n", + "T-Statistic: -13.159250585480095\n", + "P-Value: 0.0009480287588880445\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.2750470809792844\n", + "T-Statistic: -4.254446720326273\n", + "P-Value: 0.05105369953469059\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1924\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1925\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.3958333333333333\n", + "T-Statistic: -3.9531674853921452\n", + "P-Value: 0.02888393838786599\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.34597457627118644\n", + "T-Statistic: -14.428366472211776\n", + "P-Value: 0.0007217047864811893\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1925\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1926\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.38750000000000007\n", + "T-Statistic: -15.392023937439065\n", + "P-Value: 0.0005956941961595111\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.3921610169491525\n", + "T-Statistic: -5.123685756910068\n", + "P-Value: 0.014393190365221242\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1926\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1927\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.475\n", + "T-Statistic: -2.1418349190706736\n", + "P-Value: 0.12164197076622749\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3547316384180791\n", + "T-Statistic: -2.6999881415740097\n", + "P-Value: 0.07378677879364155\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1927\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1928\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3730874316939891\n", + "T-Statistic: -3.9173359194644175\n", + "P-Value: 0.029577655575744665\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.31221751412429377\n", + "T-Statistic: -4.544899514360726\n", + "P-Value: 0.01995030538814641\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1928\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1929\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6885245901639344\n", + "Average of Other Ratios: 0.3583333333333334\n", + "T-Statistic: -6.697501094028931\n", + "P-Value: 0.006790960911751886\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.34246704331450095\n", + "T-Statistic: -4.8064516129032215\n", + "P-Value: 0.04066429179543451\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1929\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1930\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4520491803278689\n", + "T-Statistic: -3.381086105792825\n", + "P-Value: 0.04305684351738099\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.3800847457627119\n", + "T-Statistic: -3.055206987855407\n", + "P-Value: 0.05520098921136872\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1930\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1931\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.34419398907103826\n", + "T-Statistic: -5.4455564685402775\n", + "P-Value: 0.012161350131513373\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.35\n", + "Average of Other Ratios: 0.2754237288135593\n", + "T-Statistic: -2.66340767777208\n", + "P-Value: 0.07611699066362566\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1931\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1932\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.75\n", + "Average of Other Ratios: 0.4474726775956285\n", + "T-Statistic: -5.428996703057984\n", + "P-Value: 0.012264667663581029\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.4429378531073446\n", + "T-Statistic: -6.248223190659335\n", + "P-Value: 0.008270584465932308\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1932\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1933\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.38333333333333336\n", + "Average of Other Ratios: 0.3279371584699453\n", + "T-Statistic: -4.7611795122867155\n", + "P-Value: 0.017592360487795486\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.2992231638418079\n", + "T-Statistic: -3.1375946196106543\n", + "P-Value: 0.05175986194673748\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1933\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1934\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.39009562841530054\n", + "T-Statistic: -7.41768759066674\n", + "P-Value: 0.005069388475696104\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.33771186440677964\n", + "T-Statistic: -7.301351957262182\n", + "P-Value: 0.005304981348263639\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1934\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1935\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.42083333333333334\n", + "T-Statistic: -1.9061102831594638\n", + "P-Value: 0.1527095539473995\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3080508474576271\n", + "T-Statistic: -3.1525851210756604\n", + "P-Value: 0.051163101927519376\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1935\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1936\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.38586065573770495\n", + "T-Statistic: -2.611459754304495\n", + "P-Value: 0.07958474323908545\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3389830508474576\n", + "Average of Other Ratios: 0.29124293785310734\n", + "T-Statistic: -3.5699799446212253\n", + "P-Value: 0.03755260022285919\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1936\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1937\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7666666666666667\n", + "Average of Other Ratios: 0.39009562841530054\n", + "T-Statistic: -41.93436311911947\n", + "P-Value: 2.9845043324434687e-05\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.38001412429378534\n", + "T-Statistic: -2.034149230295387\n", + "P-Value: 0.13480390529326824\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1937\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1938\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.40218579234972673\n", + "T-Statistic: -4.703745486314339\n", + "P-Value: 0.01818156231348825\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.3374293785310734\n", + "T-Statistic: -2.3577069273967326\n", + "P-Value: 0.09960676963753387\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1938\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1939\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -5.5469553767808515\n", + "P-Value: 0.011552909711436831\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3080508474576271\n", + "T-Statistic: -17.7463880302814\n", + "P-Value: 0.0003901203293975093\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1939\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1940\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.43333333333333335\n", + "T-Statistic: -2.4127247774526692\n", + "P-Value: 0.09478502280607064\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.3704331450094162\n", + "T-Statistic: -1.7365199955826325\n", + "P-Value: 0.22460571149425088\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1940\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1941\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.3896174863387978\n", + "T-Statistic: -6.0971062067672115\n", + "P-Value: 0.008862646607214728\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.328954802259887\n", + "T-Statistic: -5.202033883431729\n", + "P-Value: 0.013803686006307778\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1941\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1942\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.4375\n", + "T-Statistic: -2.361603838210462\n", + "P-Value: 0.09925556793924133\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3628531073446327\n", + "T-Statistic: -3.2057938752604676\n", + "P-Value: 0.04911378606623852\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1942\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1943\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7377049180327869\n", + "Average of Other Ratios: 0.4333333333333333\n", + "T-Statistic: -7.354109955820557\n", + "P-Value: 0.0051963894030398975\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.35444915254237286\n", + "T-Statistic: -3.7842912965837407\n", + "P-Value: 0.032348636596183485\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1943\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1944\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.4151639344262295\n", + "T-Statistic: -5.271696829221992\n", + "P-Value: 0.013305730606527047\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5932203389830508\n", + "Average of Other Ratios: 0.30007062146892655\n", + "T-Statistic: -5.840104693319583\n", + "P-Value: 0.010003865123379863\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1944\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1945\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.4810792349726776\n", + "T-Statistic: -3.1931249674651547\n", + "P-Value: 0.04959217968663012\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3880414312617702\n", + "T-Statistic: -2.3384829558815743\n", + "P-Value: 0.14430805754256285\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1945\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1946\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.42500000000000004\n", + "T-Statistic: -4.904097017214922\n", + "P-Value: 0.01623024607177803\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.37071563088512244\n", + "T-Statistic: -2.2572802622847377\n", + "P-Value: 0.15257812860009795\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1946\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1947\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3939890710382513\n", + "T-Statistic: -2.375708105544391\n", + "P-Value: 0.09799696855032386\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.358545197740113\n", + "T-Statistic: -2.3319342552790907\n", + "P-Value: 0.10196787590396081\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1947\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1948\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.448292349726776\n", + "T-Statistic: -5.984088746615719\n", + "P-Value: 0.00934226482517408\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.3627824858757062\n", + "T-Statistic: -1.9704993917573732\n", + "P-Value: 0.1433770749616963\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1948\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1949\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5081967213114754\n", + "Average of Other Ratios: 0.42083333333333334\n", + "T-Statistic: -2.548903055279766\n", + "P-Value: 0.08402331367690975\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4666666666666667\n", + "Average of Other Ratios: 0.3813559322033898\n", + "T-Statistic: -2.4415252920324195\n", + "P-Value: 0.09237446595146682\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1949\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1950\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.37499999999999994\n", + "T-Statistic: -6.875471004283952\n", + "P-Value: 0.006301448294158944\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3713276836158192\n", + "T-Statistic: -6.5420904956073125\n", + "P-Value: 0.007260153005952\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1950\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1951\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.375\n", + "T-Statistic: -8.04851009961339\n", + "P-Value: 0.004005903148525875\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3334745762711865\n", + "T-Statistic: -3.470097292282929\n", + "P-Value: 0.04034351536878077\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1951\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1952\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.43565573770491806\n", + "T-Statistic: -8.810845426280132\n", + "P-Value: 0.00308060726714541\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3879237288135593\n", + "T-Statistic: -2.1390171916328864\n", + "P-Value: 0.12196630264944257\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1952\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1953\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.44583333333333336\n", + "T-Statistic: -3.786983637511313\n", + "P-Value: 0.03228933418227714\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.48333333333333334\n", + "Average of Other Ratios: 0.40254237288135597\n", + "T-Statistic: -2.8010960513215193\n", + "P-Value: 0.06779122416384437\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1953\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1954\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.4066256830601093\n", + "T-Statistic: -3.9383341934243377\n", + "P-Value: 0.0291685537410846\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.379590395480226\n", + "T-Statistic: -2.325606550774779\n", + "P-Value: 0.10255792463406842\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1954\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1955\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.65\n", + "Average of Other Ratios: 0.4560792349726776\n", + "T-Statistic: -5.067243916956274\n", + "P-Value: 0.014838352365228525\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.40433145009416194\n", + "T-Statistic: -2.662987333476817\n", + "P-Value: 0.11681579707766619\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1955\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1956\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.41058743169398904\n", + "T-Statistic: -4.106153946552308\n", + "P-Value: 0.026146933323788254\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3373587570621469\n", + "T-Statistic: -5.418875755811864\n", + "P-Value: 0.012328378409982957\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1956\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1957\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.3666666666666667\n", + "T-Statistic: -11.1491127146561\n", + "P-Value: 0.0015463681392427176\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.32902542372881355\n", + "T-Statistic: -5.097322089843262\n", + "P-Value: 0.014598923701575601\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1957\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1958\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5901639344262295\n", + "Average of Other Ratios: 0.4\n", + "T-Statistic: -10.563455564617959\n", + "P-Value: 0.0018122415508561155\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.38375706214689265\n", + "T-Statistic: -2.795942882520684\n", + "P-Value: 0.06808180891277789\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1958\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1959\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.3625\n", + "T-Statistic: -5.013582256122774\n", + "P-Value: 0.015278375182884808\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3250706214689265\n", + "T-Statistic: -4.171340373881652\n", + "P-Value: 0.02508240388592301\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1959\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1960\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6557377049180327\n", + "Average of Other Ratios: 0.38333333333333336\n", + "T-Statistic: -4.064948814456438\n", + "P-Value: 0.026850007364746683\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.3246468926553673\n", + "T-Statistic: -6.228587468296307\n", + "P-Value: 0.008344518944513782\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1960\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1961\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.3569672131147541\n", + "T-Statistic: -7.573495790587439\n", + "P-Value: 0.004775004170929797\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.2838983050847458\n", + "T-Statistic: -8.87796045374059\n", + "P-Value: 0.0030132990718159855\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1961\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1962\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5573770491803278\n", + "Average of Other Ratios: 0.32500000000000007\n", + "T-Statistic: -3.590011324398241\n", + "P-Value: 0.03702281920404291\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.2824858757062147\n", + "T-Statistic: -4.417410272265133\n", + "P-Value: 0.021534399027710104\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1962\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1963\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7213114754098361\n", + "Average of Other Ratios: 0.4666666666666667\n", + "T-Statistic: -4.791779056643048\n", + "P-Value: 0.017288564339617386\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.39279661016949147\n", + "T-Statistic: -3.218304471414686\n", + "P-Value: 0.04864708009336623\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1963\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1964\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4355191256830601\n", + "T-Statistic: -2.7776540598466357\n", + "P-Value: 0.06912570353142605\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3543785310734463\n", + "T-Statistic: -3.815695684869742\n", + "P-Value: 0.03166549916342224\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1964\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1965\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5245901639344263\n", + "Average of Other Ratios: 0.325\n", + "T-Statistic: -7.221443843083816\n", + "P-Value: 0.005475227116046089\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.31228813559322033\n", + "T-Statistic: -11.326278432886252\n", + "P-Value: 0.0014762298056976652\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1965\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1966\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4687841530054645\n", + "T-Statistic: -6.1047509028321505\n", + "P-Value: 0.008831382943901647\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.4092514124293785\n", + "T-Statistic: -3.953202519249618\n", + "P-Value: 0.028883270408863487\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1966\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1967\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -12.735108311334004\n", + "P-Value: 0.0010444936119927254\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.36264124293785305\n", + "T-Statistic: -2.925487770616238\n", + "P-Value: 0.06122360955751544\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1967\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1968\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.4458333333333333\n", + "T-Statistic: -4.993743185709583\n", + "P-Value: 0.015445355698632398\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.359322033898305\n", + "T-Statistic: -1.8552679320224146\n", + "P-Value: 0.20470801229945912\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1968\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1969\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.43920765027322406\n", + "T-Statistic: -3.6522403422007157\n", + "P-Value: 0.0354367438599517\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.3965395480225989\n", + "T-Statistic: -9.383684565305357\n", + "P-Value: 0.002563738947888246\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1969\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1970\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6166666666666667\n", + "Average of Other Ratios: 0.3688524590163934\n", + "T-Statistic: -6.85232843082187\n", + "P-Value: 0.006362416633878472\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3474576271186441\n", + "T-Statistic: -5.812651983124662\n", + "P-Value: 0.010136920657992399\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1970\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1971\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.38558743169398907\n", + "T-Statistic: -3.0561029466886156\n", + "P-Value: 0.05516204355542588\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3081214689265537\n", + "T-Statistic: -3.056600083912935\n", + "P-Value: 0.05514044883654204\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1971\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1972\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.360724043715847\n", + "T-Statistic: -8.874674345400543\n", + "P-Value: 0.0030165490767111924\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3456920903954802\n", + "T-Statistic: -2.7626457956106245\n", + "P-Value: 0.06999725904230607\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1972\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1973\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.38642987249544625\n", + "T-Statistic: -3.3493937263441476\n", + "P-Value: 0.07875291250488248\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4067796610169492\n", + "Average of Other Ratios: 0.3418079096045198\n", + "T-Statistic: -4.4263520637871325\n", + "P-Value: 0.021418091531717674\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1973\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1974\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.4558743169398907\n", + "T-Statistic: -3.7027285641405947\n", + "P-Value: 0.03421325208693942\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.37528248587570623\n", + "T-Statistic: -8.13082423873902\n", + "P-Value: 0.0038896389696946266\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1974\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1975\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45901639344262296\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -2.0822726427097917\n", + "P-Value: 0.12872281264009697\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.45\n", + "Average of Other Ratios: 0.3050847457627119\n", + "T-Statistic: -8.55\n", + "P-Value: 0.0033619268709983345\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1975\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1976\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7166666666666667\n", + "Average of Other Ratios: 0.42691256830601093\n", + "T-Statistic: -7.141009375063902\n", + "P-Value: 0.005653944218972081\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3840395480225989\n", + "T-Statistic: -6.277129309215832\n", + "P-Value: 0.00816330194750534\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1976\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1977\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.42704918032786887\n", + "T-Statistic: -3.8405509086761342\n", + "P-Value: 0.031137895819023056\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4166666666666667\n", + "Average of Other Ratios: 0.3771186440677966\n", + "T-Statistic: -1.974967724404075\n", + "P-Value: 0.14275464845123892\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1977\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1978\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.44808743169398907\n", + "T-Statistic: -2.524971074527996\n", + "P-Value: 0.08580160862644565\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.36730225988700566\n", + "T-Statistic: -2.6278882027424713\n", + "P-Value: 0.07846735545649752\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1978\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1979\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6721311475409836\n", + "Average of Other Ratios: 0.5\n", + "T-Statistic: -4.543656514596028\n", + "P-Value: 0.019965007637516786\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6440677966101694\n", + "Average of Other Ratios: 0.4809322033898305\n", + "T-Statistic: -5.744562646538028\n", + "P-Value: 0.010477088755276323\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1979\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1980\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6\n", + "Average of Other Ratios: 0.5059426229508197\n", + "T-Statistic: -2.819285590304159\n", + "P-Value: 0.06677779498737416\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4576271186440678\n", + "Average of Other Ratios: 0.4137005649717514\n", + "T-Statistic: -2.715044307099425\n", + "P-Value: 0.0728534025239453\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1980\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1981\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.4125\n", + "T-Statistic: -5.2754090950771\n", + "P-Value: 0.01327985815124129\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.2867937853107345\n", + "T-Statistic: -3.7484025543573445\n", + "P-Value: 0.03315263271031142\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1981\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1982\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5\n", + "Average of Other Ratios: 0.3941256830601093\n", + "T-Statistic: -9.09710470999564\n", + "P-Value: 0.002806625908000589\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3584039548022599\n", + "T-Statistic: -3.4358556656684405\n", + "P-Value: 0.041360824576078994\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1982\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1983\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.35833333333333334\n", + "T-Statistic: -9.378324457589393\n", + "P-Value: 0.002568020999171169\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.423728813559322\n", + "Average of Other Ratios: 0.32895480225988705\n", + "T-Statistic: -3.853459154520629\n", + "P-Value: 0.03086833989707754\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1983\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1984\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.3125\n", + "T-Statistic: -10.826994016721436\n", + "P-Value: 0.0016856521372314135\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.3135593220338983\n", + "T-Statistic: -4.2613603366990604\n", + "P-Value: 0.0237024761862663\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1984\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1985\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6833333333333333\n", + "Average of Other Ratios: 0.49767759562841535\n", + "T-Statistic: -7.0718291988043145\n", + "P-Value: 0.005813854687167852\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6101694915254238\n", + "Average of Other Ratios: 0.4471751412429379\n", + "T-Statistic: -10.661446386757238\n", + "P-Value: 0.0017637451315993058\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1985\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1986\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5409836065573771\n", + "Average of Other Ratios: 0.38333333333333336\n", + "T-Statistic: -3.5333558289091114\n", + "P-Value: 0.0385464838592851\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3898305084745763\n", + "Average of Other Ratios: 0.354590395480226\n", + "T-Statistic: -2.6581313276202945\n", + "P-Value: 0.07646057355190569\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1986\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1987\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5833333333333334\n", + "Average of Other Ratios: 0.4810792349726776\n", + "T-Statistic: -2.348566398200449\n", + "P-Value: 0.10043648507703083\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5166666666666667\n", + "Average of Other Ratios: 0.42372881355932207\n", + "T-Statistic: -2.2081034216541404\n", + "P-Value: 0.11430368101220882\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1987\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1988\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.4560109289617486\n", + "T-Statistic: -2.3723659709016056\n", + "P-Value: 0.09829344194941916\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3291666666666667\n", + "T-Statistic: -4.203856330713102\n", + "P-Value: 0.024572283074481203\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1988\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1989\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.41454918032786886\n", + "T-Statistic: -2.568978311749152\n", + "P-Value: 0.08256644079807948\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5254237288135594\n", + "Average of Other Ratios: 0.400635593220339\n", + "T-Statistic: -6.380724773314517\n", + "P-Value: 0.0077935062804952064\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1989\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1990\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.7049180327868853\n", + "Average of Other Ratios: 0.4083333333333333\n", + "T-Statistic: -6.76630498961627\n", + "P-Value: 0.006595971681668088\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5084745762711864\n", + "Average of Other Ratios: 0.35861581920903957\n", + "T-Statistic: -2.999030425264594\n", + "P-Value: 0.05771345344001608\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1990\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1991\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.43333333333333335\n", + "Average of Other Ratios: 0.35252732240437157\n", + "T-Statistic: -2.7722967072440508\n", + "P-Value: 0.06943525700132411\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.3728813559322034\n", + "Average of Other Ratios: 0.295409604519774\n", + "T-Statistic: -3.600128581629112\n", + "P-Value: 0.03675886811551069\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1991\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1992\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6333333333333333\n", + "Average of Other Ratios: 0.43968579234972677\n", + "T-Statistic: -6.462196948447438\n", + "P-Value: 0.0075180098331947\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.559322033898305\n", + "Average of Other Ratios: 0.40508474576271186\n", + "T-Statistic: -8.389086394729976\n", + "P-Value: 0.003552594693743783\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1992\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1993\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6229508196721312\n", + "Average of Other Ratios: 0.3958333333333333\n", + "T-Statistic: -4.368818491249194\n", + "P-Value: 0.02218094363638083\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.3540254237288135\n", + "T-Statistic: -2.4793636360304507\n", + "P-Value: 0.08932026186301084\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1993\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1994\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6666666666666666\n", + "Average of Other Ratios: 0.4734289617486339\n", + "T-Statistic: -4.251277919668526\n", + "P-Value: 0.023852073711438902\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4915254237288136\n", + "Average of Other Ratios: 0.42189265536723164\n", + "T-Statistic: -3.2416676813361778\n", + "P-Value: 0.04779043382080003\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1994\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1995\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5737704918032787\n", + "Average of Other Ratios: 0.43333333333333335\n", + "T-Statistic: -20.63996275066481\n", + "P-Value: 0.00024870591964064987\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3673728813559322\n", + "T-Statistic: -2.8797359104311173\n", + "P-Value: 0.06354272267077887\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1995\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1996\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5333333333333333\n", + "Average of Other Ratios: 0.40232240437158473\n", + "T-Statistic: -2.8318259683509934\n", + "P-Value: 0.06609008772343258\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4\n", + "Average of Other Ratios: 0.3474576271186441\n", + "T-Statistic: -2.978383660774621\n", + "P-Value: 0.05867285324441682\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1996\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1997\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5666666666666667\n", + "Average of Other Ratios: 0.410655737704918\n", + "T-Statistic: -3.6504320440032956\n", + "P-Value: 0.03548159461405821\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.3712570621468927\n", + "T-Statistic: -2.386590270664442\n", + "P-Value: 0.09703914637860833\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1997\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1998\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.55\n", + "Average of Other Ratios: 0.44795081967213113\n", + "T-Statistic: -2.5131484576925534\n", + "P-Value: 0.08669715647138371\n", + "The highest ratio is not significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4745762711864407\n", + "Average of Other Ratios: 0.4093926553672317\n", + "T-Statistic: -2.226010109872328\n", + "P-Value: 0.11241274845462867\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1998\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 1999\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.6065573770491803\n", + "Average of Other Ratios: 0.38333333333333336\n", + "T-Statistic: -4.2005187653190665\n", + "P-Value: 0.02462402343066429\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.4406779661016949\n", + "Average of Other Ratios: 0.38834745762711864\n", + "T-Statistic: -3.1550201885686455\n", + "P-Value: 0.051066984905793825\n", + "The highest ratio is not significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 1999\n", + "___________________________________________________________________________________________________________________________\n", + "___________________________________________________________________________________________________________________________\n", + "Doing 2000\n", + "___________________________________________________________________________________________________________________________\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.639344262295082\n", + "Average of Other Ratios: 0.4041666666666667\n", + "T-Statistic: -8.814856757089686\n", + "P-Value: 0.003076528811000136\n", + "The highest ratio is significantly different from the others.\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight']\n", + "- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", + "- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Highest Match Ratio: 0.5423728813559322\n", + "Average of Other Ratios: 0.37097457627118646\n", + "T-Statistic: -5.074685831579439\n", + "P-Value: 0.014778639324226026\n", + "The highest ratio is significantly different from the others.\n", + "___________________________________________________________________________________________________________________________\n", + "Done 2000\n", + "___________________________________________________________________________________________________________________________\n" + ] + } + ], + "source": [ + "list_of_significance = []\n", + "list_of_significance_watermarked = []\n", + "count_t = 0\n", + "for text in test_cases:\n", + " count_t+=1\n", + " print(\"___________________________________________________________________________________________________________________________\")\n", + " print(\"Doing\", count_t)\n", + " print(\"___________________________________________________________________________________________________________________________\")\n", + "\n", + " words_to_add = [\"example\", \"test\", \"random\", \"insert\"]\n", + " num_words_to_add = 5\n", + "\n", + " # modified_text = randomly_add_words(text, words_to_add, num_words_to_add)\n", + " modified_text = randomly_add_words(watermark_text(text, offset=0), words_to_add, num_words_to_add)\n", + " # print(\"Original Text:\")\n", + " # print(text)\n", + " # print(\"\\nModified Text:\")\n", + " # print(modified_text)\n", + "\n", + " match_ratios = watermark_text_and_calculate_matches(modified_text, max_offset=5)\n", + " # print(match_ratios)\n", + " list_of_significance_watermarked.append(check_significant_difference(match_ratios))\n", + "\n", + " match_ratios = watermark_text_and_calculate_matches(text, max_offset=5)\n", + " list_of_significance.append(check_significant_difference(match_ratios))\n", + "\n", + " print(\"___________________________________________________________________________________________________________________________\")\n", + " print(\"Done\", count_t, )\n", + " print(\"___________________________________________________________________________________________________________________________\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": { + "id": "EHumpLgzZK0Z" + }, + "outputs": [], + "source": [ + "import pandas as pd\n", + "\n", + "df_significance = pd.DataFrame(list_of_significance, columns=['Highest Ratio', 'Average Others', 'T-Statistic', 'P-Value'])\n", + "df_significance_watermarked = pd.DataFrame(list_of_significance_watermarked, columns=['Highest Ratio', 'Average Others', 'T-Statistic', 'P-Value'])\n", + "\n", + "# Add a label column to distinguish between the two sets\n", + "df_significance['Label'] = 'Original'\n", + "df_significance_watermarked['Label'] = 'Watermarked'\n", + "\n", + "# Combine the DataFrames\n", + "combined_df = pd.concat([df_significance, df_significance_watermarked], ignore_index=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": { + "id": "I5Wm6PTHsOy-" + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Highest RatioAverage OthersT-StatisticP-ValueLabel
00.2333330.182203-3.5327580.038563Original
10.2033900.139195-3.4405910.041218Original
20.3389830.270339-2.2286080.112142Original
30.2542370.168362-2.4516130.246559Original
40.2881360.210876-5.4675400.012026Original
..................
39950.5333330.402322-2.8318260.066090Watermarked
39960.5666670.410656-3.6504320.035482Watermarked
39970.5500000.447951-2.5131480.086697Watermarked
39980.6065570.383333-4.2005190.024624Watermarked
39990.6393440.404167-8.8148570.003077Watermarked
\n", + "

4000 rows × 5 columns

\n", + "
" + ], + "text/plain": [ + " Highest Ratio Average Others T-Statistic P-Value Label\n", + "0 0.233333 0.182203 -3.532758 0.038563 Original\n", + "1 0.203390 0.139195 -3.440591 0.041218 Original\n", + "2 0.338983 0.270339 -2.228608 0.112142 Original\n", + "3 0.254237 0.168362 -2.451613 0.246559 Original\n", + "4 0.288136 0.210876 -5.467540 0.012026 Original\n", + "... ... ... ... ... ...\n", + "3995 0.533333 0.402322 -2.831826 0.066090 Watermarked\n", + "3996 0.566667 0.410656 -3.650432 0.035482 Watermarked\n", + "3997 0.550000 0.447951 -2.513148 0.086697 Watermarked\n", + "3998 0.606557 0.383333 -4.200519 0.024624 Watermarked\n", + "3999 0.639344 0.404167 -8.814857 0.003077 Watermarked\n", + "\n", + "[4000 rows x 5 columns]" + ] + }, + "execution_count": 41, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "combined_df" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": { + "id": "FHbeKfN_se6m" + }, + "outputs": [], + "source": [ + "# combined_df.to_csv(\"Results.csv\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.13" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/papers/atharva_rasane/Correlation_Matrix.png b/papers/atharva_rasane/Correlation_Matrix.png new file mode 100644 index 0000000000..e2948665f9 Binary files /dev/null and b/papers/atharva_rasane/Correlation_Matrix.png differ diff --git a/papers/atharva_rasane/Dataset.png b/papers/atharva_rasane/Dataset.png new file mode 100644 index 0000000000..a03de67c03 Binary files /dev/null and b/papers/atharva_rasane/Dataset.png differ diff --git a/papers/atharva_rasane/Distribution_of_P-value.png b/papers/atharva_rasane/Distribution_of_P-value.png new file mode 100644 index 0000000000..bdc655d742 Binary files /dev/null and b/papers/atharva_rasane/Distribution_of_P-value.png differ diff --git a/papers/atharva_rasane/Distribution_of_average_others.png b/papers/atharva_rasane/Distribution_of_average_others.png new file mode 100644 index 0000000000..3c2d1057b9 Binary files /dev/null and b/papers/atharva_rasane/Distribution_of_average_others.png differ diff --git a/papers/atharva_rasane/Distribution_of_highest_ratio.png b/papers/atharva_rasane/Distribution_of_highest_ratio.png new file mode 100644 index 0000000000..79daa9ba2e Binary files /dev/null and b/papers/atharva_rasane/Distribution_of_highest_ratio.png differ diff --git a/papers/atharva_rasane/Distribution_of_t-statistics.png b/papers/atharva_rasane/Distribution_of_t-statistics.png new file mode 100644 index 0000000000..7b464e9ad9 Binary files /dev/null and b/papers/atharva_rasane/Distribution_of_t-statistics.png differ diff --git a/papers/atharva_rasane/Results.csv b/papers/atharva_rasane/Results.csv new file mode 100644 index 0000000000..d20d5a7096 --- /dev/null +++ b/papers/atharva_rasane/Results.csv @@ -0,0 +1,4001 @@ +,Highest Ratio,Average Others,T-Statistic,P-Value,Label +0,0.23333333333333334,0.1822033898305085,-3.53275826407369,0.038562976693981454,Original +1,0.2033898305084746,0.1391949152542373,-3.4405910948750495,0.04121820653114378,Original +2,0.3389830508474576,0.27033898305084747,-2.228607614649941,0.11214158967770235,Original +3,0.2542372881355932,0.16836158192090395,-2.451612903225806,0.2465587655124727,Original +4,0.288135593220339,0.21087570621468926,-5.467540160267347,0.012025943288987453,Original +5,0.22033898305084745,0.1307909604519774,-11.145126479863883,0.0015479966208348658,Original +6,0.22033898305084745,0.1603813559322034,-4.047402698396378,0.027156785257683596,Original +7,0.3,0.22033898305084745,-11.51260179108094,0.0014069485474090153,Original +8,0.23728813559322035,0.18149717514124292,-3.7964977175244834,0.03208088709594881,Original +9,0.23728813559322035,0.16871468926553673,-3.7922455055393622,0.03217383630567124,Original +10,0.23333333333333334,0.1398305084745763,-4.975896705378727,0.015597602000975219,Original +11,0.23728813559322035,0.20261299435028246,-3.3817063110386885,0.04303714816975945,Original +12,0.288135593220339,0.18177966101694915,-3.365869501933496,0.04354367094755919,Original +13,0.22033898305084745,0.1477401129943503,-3.4073375272246085,0.04223311481214282,Original +14,0.4067796610169492,0.27838983050847455,-2.9307183932115923,0.06096526759447833,Original +15,0.16666666666666666,0.1228813559322034,-10.333333333333332,0.001933293191806968,Original +16,0.3050847457627119,0.21101694915254238,-4.384236405710172,0.02197310950253267,Original +17,0.2033898305084746,0.14745762711864407,-2.9054879908745583,0.062224127599699926,Original +18,0.23728813559322035,0.21087570621468926,-2.5909821905688375,0.08100515899541934,Original +19,0.21666666666666667,0.1440677966101695,-2.967580383634676,0.05918282683371976,Original +20,0.4067796610169492,0.3206920903954802,-4.689814564762172,0.018328326818943686,Original +21,0.2542372881355932,0.13940677966101694,-5.09630233956434,0.014606958299961344,Original +22,0.23333333333333334,0.11440677966101695,-6.328859555783819,0.007975829484392977,Original +23,0.23728813559322035,0.1896892655367232,-2.202614379084967,0.1148909616099501,Original +24,0.22033898305084745,0.12662429378531073,-5.286516953753678,0.013202833807875401,Original +25,0.3,0.1864406779661017,-2.9476070119292004,0.060140398566708664,Original +26,0.3898305084745763,0.2657485875706215,-7.979030913275075,0.004107637499059575,Original +27,0.1864406779661017,0.14336158192090395,-3.0162558762483083,0.05692805669660707,Original +28,0.21666666666666667,0.1271186440677966,-8.184904804985008,0.0038156709515571787,Original +29,0.23333333333333334,0.1822033898305085,-4.794807132575457,0.017258873277490094,Original +30,0.2711864406779661,0.13516949152542374,-9.58743044198646,0.002407749109248065,Original +31,0.2033898305084746,0.16454802259887005,-3.6666666666666683,0.03508151471548188,Original +32,0.1864406779661017,0.15176553672316384,-1.9438723809014464,0.14715294859631137,Original +33,0.2033898305084746,0.15176553672316384,-2.698151855052503,0.0739016273969179,Original +34,0.1016949152542373,0.06666666666666667,,,Original +35,0.1694915254237288,0.14058380414312618,-2.4173228346456677,0.13686029083311824,Original +36,0.3050847457627119,0.1519774011299435,-5.686705315838459,0.010777981645506028,Original +37,0.25,0.17372881355932202,-2.3302720008113575,0.10212247896202177,Original +38,0.2711864406779661,0.21949152542372882,-5.019825255742886,0.015226316305783671,Original +39,0.2711864406779661,0.1602401129943503,-3.8619097169864767,0.030693492269553303,Original +40,0.288135593220339,0.23596986817325802,-39.57142857142854,0.0006380001300463167,Original +41,0.1694915254237288,0.12238700564971752,-4.38578568651365,0.02195236472320489,Original +42,0.2542372881355932,0.1519774011299435,-5.530747598736873,0.011647446932010377,Original +43,0.21666666666666667,0.15254237288135594,-5.350441310978211,0.012770724119522098,Original +44,0.22033898305084745,0.14357344632768362,-5.128214329323895,0.01435823217533278,Original +45,0.23728813559322035,0.1646186440677966,-4.953014798968853,0.015795677695098962,Original +46,0.3050847457627119,0.17323446327683617,-5.3345252289586895,0.012876563862984138,Original +47,0.23728813559322035,0.1643361581920904,-3.2248357074853122,0.04840566051832893,Original +48,0.23728813559322035,0.16871468926553673,-3.011507892829531,0.05714319479454041,Original +49,0.3050847457627119,0.1646186440677966,-16.597491007684166,0.0004760985758523895,Original +50,0.3,0.16525423728813557,-6.7289971752910285,0.0067007729656368455,Original +51,0.2033898305084746,0.13940677966101697,-4.870967741935483,0.016533426116271753,Original +52,0.2542372881355932,0.1689265536723164,-3.6720208922977697,0.03495083324055868,Original +53,0.22033898305084745,0.15586158192090396,-2.738286769182844,0.07144110545918902,Original +54,0.22033898305084745,0.14350282485875704,-3.588902734990965,0.03705188832887151,Original +55,0.3050847457627119,0.18983050847457625,-4.577628510425044,0.01956818745991966,Original +56,0.2711864406779661,0.1853813559322034,-3.734927184999753,0.033461118399696864,Original +57,0.2542372881355932,0.21346516007532956,-7.919995572991999,0.015570889550764348,Original +58,0.23333333333333334,0.17796610169491528,-3.7720217587055536,0.03262066446770594,Original +59,0.3389830508474576,0.1815677966101695,-10.554502580376617,0.0018167602089980005,Original +60,0.2711864406779661,0.1605225988700565,-5.3922713771638495,0.012497927330704648,Original +61,0.288135593220339,0.19837570621468928,-2.935710690049308,0.06071996824652531,Original +62,0.13559322033898305,0.11789077212806026,-1.9366012620612738,0.19241125153029964,Original +63,0.3050847457627119,0.19004237288135595,-4.661502359215338,0.01863137464464403,Original +64,0.2711864406779661,0.1730225988700565,-4.393369811625047,0.021851178700442182,Original +65,0.26666666666666666,0.1822033898305085,-3.3377105216719656,0.04446314563719635,Original +66,0.22033898305084745,0.13509887005649718,-6.972220994378161,0.0060547347831298665,Original +67,0.22033898305084745,0.15607344632768363,-4.4996604055507525,0.020494569188795435,Original +68,0.2542372881355932,0.13905367231638419,-7.532784229621696,0.004849699633498134,Original +69,0.3050847457627119,0.2234463276836158,-3.191276519463654,0.049662470752465,Original +70,0.23728813559322035,0.17330508474576273,-2.1914765988605094,0.11609378562161582,Original +71,0.2542372881355932,0.19004237288135595,-3.5786014890819224,0.03732340310992795,Original +72,0.2542372881355932,0.17309322033898306,-4.574785584853341,0.019601000667008463,Original +73,0.16666666666666666,0.10593220338983052,-2.315042178850111,0.10355228938029219,Original +74,0.2711864406779661,0.21101694915254235,-5.377347857529729,0.01259437003534445,Original +75,0.23728813559322035,0.17704802259887006,-4.366099615611315,0.022217857702086435,Original +76,0.1864406779661017,0.1573446327683616,-4.756098094357983,0.04147677166903169,Original +77,0.22033898305084745,0.17309322033898306,-4.258383219097977,0.023746522626264043,Original +78,0.23728813559322035,0.14745762711864407,-2.850671138558804,0.06507315235014592,Original +79,0.22033898305084745,0.16883239171374767,-1.940875951377627,0.19179259810170837,Original +80,0.25,0.13983050847457626,-13.57805716454443,0.0008640560542882232,Original +81,0.23728813559322035,0.1687853107344633,-4.943577756944967,0.01587832283622757,Original +82,0.25,0.19491525423728814,-3.7527767497325675,0.03305327992358387,Original +83,0.2711864406779661,0.1853813559322034,-3.734927184999753,0.033461118399696864,Original +84,0.288135593220339,0.14336158192090395,-9.12238026954469,0.002784009395450553,Original +85,0.2033898305084746,0.15600282485875705,-4.763659834348825,0.017567478307811073,Original +86,0.23728813559322035,0.1515065913370998,-4.591824862480486,0.044299673534495966,Original +87,0.1864406779661017,0.1096045197740113,-9.714285714285717,0.002316933797952584,Original +88,0.288135593220339,0.2490819209039548,-3.390087084881917,0.04277212562874923,Original +89,0.288135593220339,0.19858757062146892,-2.978265932915313,0.058678380877150695,Original +90,0.2542372881355932,0.2024482109227872,-2.539664030967854,0.12632350193838743,Original +91,0.3333333333333333,0.23728813559322035,-8.01387685344754,0.004056193290243036,Original +92,0.23728813559322035,0.1434322033898305,-5.819049164593993,0.01010570903664765,Original +93,0.288135593220339,0.20218926553672317,-2.735771061149155,0.07159231928803704,Original +94,0.23728813559322035,0.13919491525423727,-4.655912421566584,0.018691975738082102,Original +95,0.2033898305084746,0.14766949152542372,-3.1966382474552573,0.04945892607281697,Original +96,0.25,0.13559322033898305,-16.53405576378645,0.00048155159575766156,Original +97,0.1864406779661017,0.13509887005649718,-2.456740106111629,0.09113124582704853,Original +98,0.2033898305084746,0.15635593220338984,-2.448992878796489,0.09176170732285532,Original +99,0.2711864406779661,0.17295197740112994,-12.697505573117574,0.0010536643393062766,Original +100,0.23728813559322035,0.18545197740112995,-2.597095416447633,0.08057786815687772,Original +101,0.2711864406779661,0.1602401129943503,-7.0060661223464,0.00597143776668511,Original +102,0.22033898305084745,0.16871468926553673,-3.3950093870826388,0.042617435426627326,Original +103,0.2033898305084746,0.1478813559322034,-2.899678131794266,0.06251860149004074,Original +104,0.13559322033898305,0.10131826741996235,-1.9782608695652164,0.1864941692932114,Original +105,0.2033898305084746,0.15628531073446328,-3.1594347385098827,0.05089331398223453,Original +106,0.3559322033898305,0.29964689265536726,-10.350649350649334,0.001923817806020011,Original +107,0.2033898305084746,0.14766949152542372,-5.257547050264664,0.013404952501338205,Original +108,0.22033898305084745,0.15190677966101696,-2.742051411140234,0.07121556090757529,Original +109,0.3333333333333333,0.17796610169491528,-7.701540462154052,0.004549748975956458,Original +110,0.23728813559322035,0.10557909604519773,-7.476466358952792,0.004955591570430506,Original +111,0.2033898305084746,0.11843220338983051,-3.149160708078649,0.05129865697051939,Original +112,0.23728813559322035,0.1602401129943503,-2.951009970239908,0.05997588776618918,Original +113,0.2033898305084746,0.18559322033898307,-2.848958479370646,0.06516476187287569,Original +114,0.1694915254237288,0.11798493408662901,-5.251610061723054,0.034398946176199485,Original +115,0.4,0.24152542372881358,-10.949598818482546,0.001630748987105004,Original +116,0.2542372881355932,0.15593220338983052,-6.457745685519285,0.007532728000207892,Original +117,0.1864406779661017,0.13926553672316386,-5.731425162505742,0.010544436415387572,Original +118,0.1864406779661017,0.16299435028248588,-3.608695652173914,0.06894253641177729,Original +119,0.3220338983050847,0.1518361581920904,-9.402305491422489,0.0025489364534890947,Original +120,0.25,0.211864406779661,-7.794228634059958,0.004395375691816533,Original +121,0.26666666666666666,0.24152542372881355,-3.098582276011423,0.05335457237433866,Original +122,0.3,0.24152542372881355,-3.1118145559317116,0.052806824094954664,Original +123,0.23333333333333334,0.17372881355932202,-2.8133333333333344,0.0671073341823401,Original +124,0.288135593220339,0.17274011299435027,-4.913402497837348,0.016146391435118687,Original +125,0.1694915254237288,0.13926553672316386,-3.6722304933420005,0.03494573014722615,Original +126,0.2711864406779661,0.21497175141242938,-2.7808379940637797,0.06894254556283926,Original +127,0.2542372881355932,0.21115819209039546,-2.8047829882173874,0.06758426686914822,Original +128,0.23333333333333334,0.19067796610169493,-5.2571452098620695,0.01340778439483409,Original +129,0.2033898305084746,0.11822033898305086,-6.986639340848923,0.0060190635058901916,Original +130,0.2033898305084746,0.16045197740112996,-3.771816669089535,0.03262523637568915,Original +131,0.2033898305084746,0.1605225988700565,-2.828645932579055,0.0662636402779568,Original +132,0.15254237288135594,0.10103578154425613,-3.0973237391013955,0.0903361791294822,Original +133,0.3050847457627119,0.1434322033898305,-9.210615078582297,0.0027069229317899495,Original +134,0.288135593220339,0.1896186440677966,-4.894736842105265,0.016315166384743927,Original +135,0.3559322033898305,0.23213276836158192,-5.049357722281286,0.014983160961383876,Original +136,0.2542372881355932,0.2236581920903955,-3.6741121386261355,0.03489996095058197,Original +137,0.3050847457627119,0.2531779661016949,-2.8305893057056326,0.06615751198285083,Original +138,0.2542372881355932,0.23587570621468926,-2.137186834969644,0.16604961117054062,Original +139,0.18333333333333332,0.11864406779661017,-3.5335467141319046,0.038541217308166585,Original +140,0.2033898305084746,0.12662429378531073,-5.193353369212653,0.013867428943147485,Original +141,0.2542372881355932,0.1434322033898305,-6.313436023720237,0.00803112776427947,Original +142,0.2542372881355932,0.15600282485875705,-3.541825936051609,0.03831367266160501,Original +143,0.3050847457627119,0.16440677966101697,-8.34181386665146,0.00361131585696273,Original +144,0.2033898305084746,0.1348870056497175,-6.050693757052515,0.009055606464007803,Original +145,0.22033898305084745,0.18135593220338983,-5.276561879022918,0.01327183724912819,Original +146,0.26666666666666666,0.17796610169491525,-5.028024029479735,0.015158299248004988,Original +147,0.26666666666666666,0.1822033898305085,-7.92070349524896,0.004195693967370819,Original +148,0.23728813559322035,0.18545197740112995,-2.9798032437751925,0.05860625147640999,Original +149,0.3728813559322034,0.26958568738229755,-2.957002218815846,0.09786519767422906,Original +150,0.23333333333333334,0.1652542372881356,-3.6229338549736347,0.03617265345530896,Original +151,0.2711864406779661,0.22796610169491527,-2.846542418148333,0.06529426992156556,Original +152,0.2711864406779661,0.14774011299435028,-4.041966945913288,0.02725273967177257,Original +153,0.2542372881355932,0.19406779661016949,-3.4332517325533063,0.041439515376910444,Original +154,0.2711864406779661,0.14738700564971752,-4.874543567127261,0.016500349284977168,Original +155,0.23333333333333334,0.211864406779661,-4.387862045841163,0.021924602127579518,Original +156,0.23728813559322035,0.1796610169491525,-5.666666666666667,0.029758752589905717,Original +157,0.23728813559322035,0.19830508474576272,-3.682947537517003,0.034686070852458215,Original +158,0.22033898305084745,0.15160075329566855,-4.166330062408052,0.05306537932277536,Original +159,0.23728813559322035,0.19392655367231637,-2.224345699469886,0.11258691448891019,Original +160,0.1694915254237288,0.1056497175141243,-4.336541993961348,0.022624336731357778,Original +161,0.23728813559322035,0.1797551789077213,-3.8907727779580643,0.060159036553398035,Original +162,0.2542372881355932,0.18968926553672316,-2.986928104575163,0.058273407134112075,Original +163,0.22033898305084745,0.14048964218455745,-3.229591652487888,0.08397470085153524,Original +164,0.1694915254237288,0.1391949152542373,-3.8838243353571547,0.03024595201311427,Original +165,0.2542372881355932,0.16885593220338982,-3.6939328104988762,0.03442246894212467,Original +166,0.2542372881355932,0.20268361581920902,-3.2230287355249403,0.04847230125929893,Original +167,0.2542372881355932,0.22201351933310776,-3.1365710180077406,0.0518009301559042,Original +168,0.26666666666666666,0.21610169491525424,-3.493722261155749,0.0396601427679115,Original +169,0.23728813559322035,0.1910546139359699,-7.418137270026101,0.017691506692045566,Original +170,0.3050847457627119,0.23192090395480225,-3.3773352617852765,0.043176200293171145,Original +171,0.2033898305084746,0.1432909604519774,-1.8928833055825962,0.15471358909740393,Original +172,0.2033898305084746,0.1290018832391714,-3.8230779561170367,0.06211218967841154,Original +173,0.3050847457627119,0.2193502824858757,-4.75599598618577,0.017644508859181555,Original +174,0.2033898305084746,0.1601694915254237,-2.0803333919424123,0.12896148504661395,Original +175,0.288135593220339,0.1901129943502825,-3.182414988821109,0.05000120219940348,Original +176,0.3389830508474576,0.2531779661016949,-6.182185493474629,0.008522718517249426,Original +177,0.3559322033898305,0.2658898305084746,-4.202979499690264,0.024585862477064943,Original +178,0.288135593220339,0.2192090395480226,-3.4729797480883895,0.040259338911251656,Original +179,0.3050847457627119,0.19399717514124296,-7.840633887955942,0.004320681154632681,Original +180,0.288135593220339,0.19413841807909604,-5.2860346002987235,0.013206166418066021,Original +181,0.2711864406779661,0.2078154425612053,-2.134529747722321,0.16636576065135147,Original +182,0.2711864406779661,0.22372881355932203,-3.59486813709167,0.036895807895617604,Original +183,0.3220338983050847,0.12245762711864407,-7.84890994931086,0.004307536410571643,Original +184,0.2833333333333333,0.21610169491525422,-2.401102376173316,0.09577949808406833,Original +185,0.2542372881355932,0.20254237288135596,-7.415534221028932,0.005073622594306211,Original +186,0.2711864406779661,0.1772598870056497,-6.333333333333335,0.007959883216421762,Original +187,0.18333333333333332,0.15677966101694915,-6.26666666666667,0.00820192086968827,Original +188,0.2542372881355932,0.19413841807909604,-5.382088936904238,0.012563625893657768,Original +189,0.2711864406779661,0.23545197740112994,-1.9022556390977454,0.30811702486531156,Original +190,0.35,0.2754237288135593,-9.191300234460838,0.0027235534322141448,Original +191,0.26666666666666666,0.20338983050847456,-2.536300556483895,0.08495405146875473,Original +192,0.23728813559322035,0.1307909604519774,-10.043517801177494,0.002101488368509634,Original +193,0.22033898305084745,0.14766949152542375,-6.856800905858473,0.006350573689746284,Original +194,0.288135593220339,0.16871468926553673,-7.1482687865653824,0.005637501684561561,Original +195,0.23333333333333334,0.1483050847457627,-2.948006683383839,0.06012104778921462,Original +196,0.26666666666666666,0.14830508474576273,-4.1037036556074025,0.02618807381282589,Original +197,0.22033898305084745,0.16885593220338985,-6.912825719494809,0.0062046516135269335,Original +198,0.21666666666666667,0.1694915254237288,-3.936227748605116,0.02920926328962092,Original +199,0.288135593220339,0.19858757062146892,-3.923159163483533,0.029463468310785026,Original +200,0.4745762711864407,0.3416666666666667,-7.374780272477125,0.0051546431499948815,Original +201,0.4406779661016949,0.346045197740113,-6.302829818170098,0.008069446993235066,Original +202,0.5423728813559322,0.38834745762711864,-7.130307147698573,0.005678300191929151,Original +203,0.4576271186440678,0.36292372881355933,-3.837369345431411,0.031204798309479358,Original +204,0.4745762711864407,0.37542372881355934,-7.414573731136387,0.005075512686812505,Original +205,0.43333333333333335,0.3686440677966102,-4.46962233410428,0.020866618588713408,Original +206,0.4666666666666667,0.3601694915254237,-7.358286550031938,0.005187918406325382,Original +207,0.4745762711864407,0.3504237288135593,-6.6711622996447275,0.006867611409633449,Original +208,0.3898305084745763,0.3543785310734463,-3.127498225142409,0.05216674452503871,Original +209,0.3898305084745763,0.3331920903954802,-1.7611959878594572,0.1764230443495778,Original +210,0.5254237288135594,0.4387005649717514,-3.5958734600175015,0.03686958706248058,Original +211,0.4576271186440678,0.3257062146892655,-6.924029541793238,0.02022773333106733,Original +212,0.4576271186440678,0.3334745762711865,-4.989086378646126,0.015484895041677766,Original +213,0.4406779661016949,0.29555084745762716,-4.560758399398884,0.019763952369741263,Original +214,0.4745762711864407,0.37570621468926557,-3.448652473575024,0.04097685603288608,Original +215,0.4406779661016949,0.388135593220339,-4.65582342119235,0.018692942668843485,Original +216,0.4406779661016949,0.37125706214689264,-4.6417007257391365,0.018847202898111523,Original +217,0.4666666666666667,0.3008474576271187,-3.3764082011477075,0.04320576532803089,Original +218,0.4745762711864407,0.3799435028248587,-2.299434047232845,0.10504286102353275,Original +219,0.4745762711864407,0.3674435028248587,-3.575513903469711,0.03740527845126834,Original +220,0.4745762711864407,0.4132768361581921,-2.677380542667445,0.07521629738306508,Original +221,0.5166666666666667,0.3559322033898305,-8.779860612843027,0.0031123529183814187,Original +222,0.4406779661016949,0.38210922787193974,-2.7837850452128095,0.10845055325071953,Original +223,0.4915254237288136,0.35812146892655367,-2.5160267825812963,0.08647807452830744,Original +224,0.4166666666666667,0.3601694915254237,-3.233808333817773,0.0480764627944047,Original +225,0.4406779661016949,0.36694915254237287,-2.474962294633946,0.08966910755373639,Original +226,0.4745762711864407,0.3294491525423729,-5.302012165253797,0.013096358284056045,Original +227,0.4406779661016949,0.3333333333333333,-4.82600482600724,0.016956798300543117,Original +228,0.4666666666666667,0.3177966101694915,-7.026666666666668,0.0059214753537602605,Original +229,0.5084745762711864,0.379590395480226,-4.111518045317934,0.02605716031917639,Original +230,0.4576271186440678,0.41007532956685494,-4.605203601134296,0.04405939035816047,Original +231,0.5084745762711864,0.40127118644067794,-2.4156666462852487,0.09453530050793599,Original +232,0.4666666666666667,0.3686440677966102,-3.612819691689752,0.036431155449573864,Original +233,0.5254237288135594,0.36299435028248583,-4.890297438969469,0.016355645150575224,Original +234,0.4406779661016949,0.3626412429378531,-2.3461567951496813,0.1006566092112587,Original +235,0.4915254237288136,0.41035781544256117,-3.6256912791125826,0.06836231728156876,Original +236,0.6779661016949152,0.413135593220339,-6.782402329135958,0.006551418920014316,Original +237,0.5932203389830508,0.40091807909604515,-7.503655577513371,0.004904092057273042,Original +238,0.423728813559322,0.3500706214689266,-4.153796413823909,0.025363321264457474,Original +239,0.5254237288135594,0.37645951035781544,-9.627032312606932,0.010618294541056795,Original +240,0.3728813559322034,0.307909604519774,-2.524577979762879,0.08583120198777253,Original +241,0.5,0.423728813559322,-2.5287816912705736,0.08551538240691227,Original +242,0.4576271186440678,0.32478813559322034,-11.455677986899559,0.0014276482471677783,Original +243,0.43333333333333335,0.3050847457627119,-5.140537735016466,0.01426366105655198,Original +244,0.43333333333333335,0.3813559322033898,-1.4070831677394464,0.25411009793550343,Original +245,0.4576271186440678,0.32083333333333336,-3.313513183532759,0.045273022622265666,Original +246,0.4406779661016949,0.35056497175141244,-2.7353018723026903,0.07162056517888178,Original +247,0.4576271186440678,0.3629943502824859,-5.564047150200574,0.011454300879796521,Original +248,0.5084745762711864,0.3461158192090395,-16.539568345323744,0.0004810744367490006,Original +249,0.3898305084745763,0.3501412429378531,-2.5736529074083476,0.08223167628896547,Original +250,0.4576271186440678,0.354590395480226,-4.788344485301914,0.01732232163571842,Original +251,0.43333333333333335,0.3728813559322034,-2.9121711386422233,0.061887533056045364,Original +252,0.4745762711864407,0.3759887005649718,-2.087127965330441,0.17214627140969266,Original +253,0.5333333333333333,0.3898305084745763,-5.7519673334545445,0.010439377744934933,Original +254,0.4406779661016949,0.35882768361581924,-4.431514865270233,0.021351308327048055,Original +255,0.45,0.35169491525423735,-2.667077795032928,0.07587912763738508,Original +256,0.4,0.32627118644067793,-2.8103535287436263,0.06727306836232257,Original +257,0.4406779661016949,0.3484934086629002,-4.258091646437124,0.0509728994156799,Original +258,0.4576271186440678,0.3923022598870056,-4.900769721140662,0.01626036695364969,Original +259,0.3898305084745763,0.28700564971751413,-6.458938312987853,0.007528780847255386,Original +260,0.423728813559322,0.3671610169491525,-3.7775875209558483,0.032496904050512185,Original +261,0.4745762711864407,0.40084745762711865,-5.117647058823531,0.014439978840531617,Original +262,0.4576271186440678,0.31631355932203387,-4.390851056341401,0.021884716395917995,Original +263,0.3333333333333333,0.2711864406779661,-4.016632088371216,0.02770576700479268,Original +264,0.4406779661016949,0.3540489642184557,-3.260853543038344,0.08256523104344153,Original +265,0.4576271186440678,0.37563559322033896,-4.895597481406331,0.016307334114510047,Original +266,0.5084745762711864,0.39682203389830506,-4.774692154738101,0.017457356531658943,Original +267,0.5166666666666667,0.3771186440677966,-6.9688146846829495,0.00606320270460328,Original +268,0.4745762711864407,0.40480225988700563,-2.244619924618763,0.15392599376576382,Original +269,0.4067796610169492,0.33785310734463275,-2.752891462679799,0.07057104088631745,Original +270,0.45,0.32627118644067793,-6.584419205304784,0.007128240934046268,Original +271,0.4067796610169492,0.33340395480225987,-2.392200090989989,0.09654983618738305,Original +272,0.4067796610169492,0.3500706214689266,-2.147266074887937,0.12101972629670502,Original +273,0.4915254237288136,0.39896421845574387,-4.392202321683557,0.048125248972546525,Original +274,0.4067796610169492,0.363135593220339,-2.037441371070689,0.1343771965437019,Original +275,0.48333333333333334,0.3601694915254237,-4.043803171028667,0.027220277010713455,Original +276,0.4067796610169492,0.32916666666666666,-4.334673395356122,0.0226503559908631,Original +277,0.4166666666666667,0.37711864406779666,-1.9749677244040724,0.14275464845123928,Original +278,0.5254237288135594,0.4472457627118644,-4.483251319120438,0.020696735526511162,Original +279,0.4915254237288136,0.346045197740113,-5.143574545220902,0.014240481118726733,Original +280,0.4576271186440678,0.29971751412429376,-8.707841203347302,0.0031878312735672807,Original +281,0.4745762711864407,0.3206920903954802,-6.159177470925122,0.008612932023026598,Original +282,0.3898305084745763,0.3248587570621469,-5.276561879022925,0.01327183724912815,Original +283,0.4406779661016949,0.34597457627118644,-4.470968814876207,0.020849754679967962,Original +284,0.5084745762711864,0.34590395480225994,-3.714817178281528,0.033928358727363876,Original +285,0.4576271186440678,0.3923728813559322,-2.707175832648326,0.07333935451425552,Original +286,0.3898305084745763,0.3082627118644068,-3.1270512783828903,0.052184849438917776,Original +287,0.423728813559322,0.37617702448210927,-2.1147485270709563,0.16874539562177085,Original +288,0.5423728813559322,0.36292372881355933,-5.215987959681293,0.013702017252772999,Original +289,0.4576271186440678,0.3816384180790961,-2.042749961024093,0.17780819467640524,Original +290,0.43333333333333335,0.3559322033898305,-2.3848638865930605,0.09719033554345949,Original +291,0.4067796610169492,0.3163841807909605,-13.063945294843638,0.0009686387721898685,Original +292,0.3898305084745763,0.346045197740113,-7.51860437612632,0.0048760779327184315,Original +293,0.4576271186440678,0.35,-5.823074704906403,0.010086133108957392,Original +294,0.5084745762711864,0.4718455743879473,-2.3238483653527293,0.145752005957584,Original +295,0.423728813559322,0.38747645951035786,-2.48387096774193,0.13098307174005927,Original +296,0.4915254237288136,0.3967514124293785,-6.064766311184656,0.008996522307758008,Original +297,0.4406779661016949,0.3672316384180791,-3.2003787654626485,0.04931755050578723,Original +298,0.423728813559322,0.3670197740112995,-2.739338279816391,0.07137801893519374,Original +299,0.4745762711864407,0.3377118644067797,-8.333345166769199,0.0036219707869269407,Original +300,0.4166666666666667,0.3432203389830508,-3.466666666666669,0.04044399231953359,Original +301,0.4745762711864407,0.3757062146892655,-6.28618557093712,0.00813006680248645,Original +302,0.4745762711864407,0.3204802259887006,-5.96246084588649,0.00943790429103949,Original +303,0.5,0.3644067796610169,-6.357755313912211,0.00787356840736677,Original +304,0.43333333333333335,0.3771186440677966,-1.5819079806991574,0.21181729689488749,Original +305,0.5166666666666667,0.4067796610169492,-4.404557662822478,0.021703011848884367,Original +306,0.4745762711864407,0.4101694915254237,-2.577629100382969,0.12328440517539614,Original +307,0.5084745762711864,0.3753531073446328,-6.31056255621062,0.008041485679774507,Original +308,0.4576271186440678,0.3672316384180791,-7.5424723326565015,0.00483178537362738,Original +309,0.423728813559322,0.3145951035781544,-3.1849674947750137,0.08604712307024512,Original +310,0.3898305084745763,0.32551789077212806,-1.4349594508232855,0.2877633918978512,Original +311,0.4745762711864407,0.354590395480226,-3.4224483105094405,0.041768035740251584,Original +312,0.5084745762711864,0.3714689265536723,-5.34778311027372,0.012788321514374179,Original +313,0.48333333333333334,0.3771186440677966,-2.881670261299946,0.06344247147107336,Original +314,0.423728813559322,0.33749999999999997,-14.93808506039733,0.0006510638055588633,Original +315,0.4915254237288136,0.3628531073446327,-7.3695757821568195,0.005165112471402641,Original +316,0.3898305084745763,0.32937853107344633,-2.297491967496297,0.10523014082412589,Original +317,0.4745762711864407,0.42627118644067796,-4.07489509276445,0.02667808711215743,Original +318,0.4745762711864407,0.3206920903954802,-6.4099048762033455,0.00769333223799094,Original +319,0.288135593220339,0.2196327683615819,-3.2102314527592526,0.04894759775912512,Original +320,0.5254237288135594,0.3799435028248588,-4.547561092589984,0.019918871326456516,Original +321,0.3898305084745763,0.3415960451977401,-2.7606608826884482,0.07011354688774671,Original +322,0.3898305084745763,0.33778248587570625,-2.679574844992895,0.07507605335291008,Original +323,0.4576271186440678,0.36687853107344637,-2.7295669647405267,0.07196693599133859,Original +324,0.5084745762711864,0.3924435028248588,-4.124951949583921,0.02583406935919925,Original +325,0.4406779661016949,0.3293079096045197,-5.9823754058047705,0.009349794795095102,Original +326,0.4406779661016949,0.3120056497175141,-4.032355280571854,0.027423480466249588,Original +327,0.559322033898305,0.3836864406779661,-4.836938005621383,0.01685256521081953,Original +328,0.4406779661016949,0.3209745762711864,-4.989071878278333,0.015485018365118487,Original +329,0.43333333333333335,0.3389830508474576,-6.817746450746516,0.00645497746167015,Original +330,0.4067796610169492,0.3292372881355932,-5.0198252557428855,0.015226316305783671,Original +331,0.3898305084745763,0.3586158192090395,-4.20590123701231,0.0245406514009237,Original +332,0.4745762711864407,0.36730225988700566,-5.386626427441342,0.012534293739746833,Original +333,0.5,0.385593220338983,-9.000000000000007,0.002895812161864139,Original +334,0.5084745762711864,0.3543785310734463,-13.59402614992177,0.0008610534762829226,Original +335,0.3898305084745763,0.2635593220338983,-2.8494467148061093,0.10425518121888491,Original +336,0.423728813559322,0.3543785310734463,-12.120686030907725,0.0012087821339881704,Original +337,0.45,0.34745762711864403,-9.372619697821948,0.00257258879538172,Original +338,0.576271186440678,0.3879943502824859,-9.187860980455177,0.0027265288821396168,Original +339,0.4745762711864407,0.3543079096045198,-3.2679435909119663,0.046849662447178565,Original +340,0.5254237288135594,0.4048728813559322,-6.267903613009531,0.008197342653735116,Original +341,0.3898305084745763,0.3246468926553672,-2.4329184228681733,0.0930869236120761,Original +342,0.45,0.364406779661017,-7.823426359338976,0.004348181653136324,Original +343,0.4576271186440678,0.36313559322033895,-3.4592703994453387,0.04066170197907394,Original +344,0.3898305084745763,0.27429378531073445,-13.77956011482232,0.0008271633740114473,Original +345,0.4406779661016949,0.34562146892655365,-2.1945120664549007,0.11576447779308814,Original +346,0.4576271186440678,0.37535310734463284,-2.7799229360270217,0.06899512300862856,Original +347,0.3728813559322034,0.28691148775894537,-2.3579139690812547,0.14242127080654132,Original +348,0.4406779661016949,0.328954802259887,-8.470193664445993,0.003454751941090446,Original +349,0.4666666666666667,0.4110169491525424,-3.8450462874172215,0.031043679643556813,Original +350,0.4745762711864407,0.3502824858757062,-8.315218406202995,0.003644917117727713,Original +351,0.4576271186440678,0.3632062146892655,-2.5878167974480393,0.08122750520425058,Original +352,0.5166666666666667,0.4110169491525424,-9.907502030846539,0.0021871339690660045,Original +353,0.4576271186440678,0.417725988700565,-3.7729601826121053,0.03259975536350789,Original +354,0.4915254237288136,0.3757062146892655,-6.25244138363679,0.00825481406798463,Original +355,0.423728813559322,0.3332627118644068,-9.283339926398568,0.0026454980508478078,Original +356,0.4915254237288136,0.36320621468926556,-4.276430704251948,0.023481122723639215,Original +357,0.5084745762711864,0.40543785310734465,-3.1308652437968307,0.05203060761024695,Original +358,0.4576271186440678,0.3670197740112995,-3.3865448130535376,0.04288388807797634,Original +359,0.4745762711864407,0.3963983050847458,-2.355804437045393,0.09977877653717733,Original +360,0.4067796610169492,0.2953389830508475,-13.691233125567791,0.000843071969469656,Original +361,0.4666666666666667,0.3983050847457627,-3.3886778033477842,0.042816545584287505,Original +362,0.5166666666666667,0.3855932203389831,-3.8368077899595163,0.03121662594701848,Original +363,0.5254237288135594,0.30430790960451976,-4.336743966663418,0.022621526682653798,Original +364,0.4745762711864407,0.35007062146892653,-7.021230179838503,0.005934606781342949,Original +365,0.4915254237288136,0.37521186440677967,-3.216398138197251,0.04871783239364945,Original +366,0.423728813559322,0.3418079096045198,-3.841143835488625,0.031125448001670088,Original +367,0.423728813559322,0.34187853107344635,-3.1832852004313463,0.04996780945878653,Original +368,0.5,0.3686440677966102,-5.6286160538198615,0.011091567639778482,Original +369,0.4406779661016949,0.3332627118644068,-6.9539103841401015,0.0061004380937106,Original +370,0.4745762711864407,0.37584745762711863,-2.9318808768994544,0.06090803764359439,Original +371,0.559322033898305,0.4004237288135593,-3.7578369991437737,0.03293881990782305,Original +372,0.4576271186440678,0.37966101694915255,-5.662208585049302,0.010908789440745359,Original +373,0.4576271186440678,0.3548022598870057,-3.46808249954851,0.04040248651626954,Original +374,0.4406779661016949,0.3711158192090396,-2.4516068885038713,0.09154838744355702,Original +375,0.5084745762711864,0.36271186440677966,-9.86487655643415,0.0022149241738372996,Original +376,0.4576271186440678,0.3713983050847458,-8.050860563356986,0.0040025200155213245,Original +377,0.423728813559322,0.32507062146892657,-2.8491875970653315,0.06515249712984865,Original +378,0.4915254237288136,0.36306497175141245,-6.0781754491615825,0.008940693333973865,Original +379,0.5166666666666667,0.4406779661016949,-5.490939506738963,0.011883974620146622,Original +380,0.559322033898305,0.4046610169491526,-3.190034610277395,0.04970976762287175,Original +381,0.4406779661016949,0.33375706214689266,-2.933998334563753,0.06080396633288608,Original +382,0.4406779661016949,0.3626412429378531,-2.0904012105524146,0.1277280900520854,Original +383,0.4745762711864407,0.3504237288135593,-5.129835675678766,0.014345743161730394,Original +384,0.4067796610169492,0.3543079096045198,-4.181753231671517,0.02491756822152362,Original +385,0.423728813559322,0.346045197740113,-2.667891875399661,0.07582649117357966,Original +386,0.4915254237288136,0.3714689265536723,-5.879572942861632,0.009816556345478428,Original +387,0.5084745762711864,0.37966101694915255,-6.598309513974847,0.007085641181587867,Original +388,0.4745762711864407,0.3627824858757063,-4.484419263456089,0.020682260794596567,Original +389,0.559322033898305,0.3836864406779661,-4.199186687635008,0.024644713133568197,Original +390,0.4406779661016949,0.35444915254237286,-7.140350877192979,0.0056554388389449695,Original +391,0.4406779661016949,0.3755649717514124,-4.427578329769953,0.021402204751343277,Original +392,0.43333333333333335,0.37288135593220345,-2.6341579069577183,0.0780460308657842,Original +393,0.4067796610169492,0.33719397363465164,-2.622926709469525,0.1197907664196296,Original +394,0.5084745762711864,0.36299435028248583,-8.553684424935213,0.003357721498987871,Original +395,0.4576271186440678,0.33326271186440676,-18.123845744121766,0.00036642093322613114,Original +396,0.5333333333333333,0.36864406779661013,-3.142183764359604,0.051576243518155256,Original +397,0.5,0.3389830508474576,-7.756717518813398,0.004457004963920533,Original +398,0.4745762711864407,0.3290960451977401,-5.1956627047447785,0.013850433541822478,Original +399,0.423728813559322,0.3206920903954802,-4.95328158335475,0.015793349475275394,Original +400,0.4745762711864407,0.4135593220338983,-3.3916011823151724,0.04272446714466854,Original +401,0.4745762711864407,0.3248587570621469,-6.203180801394921,0.008441476456348773,Original +402,0.5254237288135594,0.40077683615819215,-5.011757995152538,0.015293631034164368,Original +403,0.4745762711864407,0.39237288135593223,-11.151144229232496,0.0015455390866235931,Original +404,0.4067796610169492,0.28714689265536725,-3.409092533983689,0.04217876250784662,Original +405,0.423728813559322,0.33173258003766476,-3.524320965413247,0.07193114099352556,Original +406,0.43333333333333335,0.3305084745762712,-2.3067562719090997,0.10434037801243047,Original +407,0.4666666666666667,0.36016949152542377,-2.4527621833439794,0.09145430123164801,Original +408,0.4067796610169492,0.3483050847457627,-11.775818566563489,0.00713429023847155,Original +409,0.423728813559322,0.38806497175141247,-2.8670514268084752,0.06420504585478538,Original +410,0.4406779661016949,0.4092514124293785,-2.568138862870189,0.08262673431862806,Original +411,0.4745762711864407,0.3839689265536724,-2.903867111427604,0.06230610742939453,Original +412,0.4666666666666667,0.3940677966101695,-6.80809631531433,0.006481122703408014,Original +413,0.4406779661016949,0.3542372881355932,-2.2861298780156964,0.10633394781548948,Original +414,0.4745762711864407,0.38425141242937855,-2.687063737163629,0.07459984908288483,Original +415,0.4406779661016949,0.3536723163841808,-3.172729727206767,0.08662826593910922,Original +416,0.5166666666666667,0.35593220338983045,-4.843649660017283,0.016788991301625738,Original +417,0.4406779661016949,0.3670197740112994,-2.9579994448141673,0.05963975694231845,Original +418,0.423728813559322,0.30903954802259886,-2.5908795387845776,0.12224781736312054,Original +419,0.576271186440678,0.4008474576271186,-8.297720726530276,0.0036672497469535698,Original +420,0.4406779661016949,0.38182674199623357,-1.9595943025088522,0.1891147904640091,Original +421,0.3898305084745763,0.32076271186440675,-4.322021374340152,0.022827550876124904,Original +422,0.4745762711864407,0.4388418079096046,-2.947245022552593,0.0601579316359931,Original +423,0.4915254237288136,0.39632768361581916,-4.060240963855425,0.02693187975555432,Original +424,0.3898305084745763,0.3418079096045198,-3.27165152540788,0.04671879927774851,Original +425,0.3898305084745763,0.3483992467043315,-5.499999999999996,0.031504003041813854,Original +426,0.5833333333333334,0.4279661016949152,-10.734900802433872,0.0017285150870629262,Original +427,0.5084745762711864,0.34230225988700563,-3.432024351211093,0.041476672849086224,Original +428,0.5254237288135594,0.35021186440677965,-10.964467451419276,0.0016242528673193293,Original +429,0.5254237288135594,0.37577683615819213,-5.036096342175906,0.015091717648582546,Original +430,0.4915254237288136,0.4134887005649718,-4.473097699753992,0.020823127617620334,Original +431,0.3559322033898305,0.3081214689265537,-3.5637008822210965,0.037720655653052076,Original +432,0.5084745762711864,0.37563559322033896,-4.420794483496409,0.02149028401764827,Original +433,0.5166666666666667,0.34745762711864403,-5.845641436961716,0.009977307916285254,Original +434,0.4576271186440678,0.3375706214689265,-16.999999999999993,0.00044334353831207803,Original +435,0.4915254237288136,0.4263418079096045,-4.09478757977338,0.026338481224541917,Original +436,0.4406779661016949,0.3378531073446328,-3.7042710296105374,0.034176730688008236,Original +437,0.4745762711864407,0.39661016949152544,-3.8969992136018448,0.029980924038161237,Original +438,0.45,0.3305084745762712,-3.6406043454349715,0.03572662155623015,Original +439,0.5084745762711864,0.42203389830508475,-6.715954603984051,0.006737927535009723,Original +440,0.4067796610169492,0.32916666666666666,-13.44549998474748,0.0008895256433019566,Original +441,0.3898305084745763,0.3208333333333333,-3.6177990004181297,0.0363035983070319,Original +442,0.43333333333333335,0.37288135593220345,-2.5220141862320173,0.08602451986599671,Original +443,0.4666666666666667,0.3940677966101695,-5.016126374955461,0.01525713242925573,Original +444,0.4915254237288136,0.39223163841807906,-3.5593682026592988,0.03783717712998915,Original +445,0.559322033898305,0.41744350282485876,-3.7183243247458044,0.03384627426695061,Original +446,0.5,0.35593220338983056,-4.164132562731401,0.025197328469594847,Original +447,0.559322033898305,0.37584745762711863,-5.567318968997769,0.01143555025754969,Original +448,0.5084745762711864,0.43036723163841806,-3.6908520053100666,0.03449613636026554,Original +449,0.5084745762711864,0.4135593220338983,-6.287241983947008,0.008126201507413536,Original +450,0.423728813559322,0.354590395480226,-4.195847013586463,0.024696683385781026,Original +451,0.559322033898305,0.3545197740112994,-7.100680549678681,0.0057464492134181155,Original +452,0.4406779661016949,0.3925141242937853,-2.652993667727985,0.07679697042443225,Original +453,0.559322033898305,0.32492937853107345,-10.457966297708378,0.0018664458731184775,Original +454,0.423728813559322,0.35868644067796607,-13.182726027622682,0.000943041742019242,Original +455,0.43333333333333335,0.3855932203389831,-4.476919366879844,0.0207754384968935,Original +456,0.3559322033898305,0.3038135593220339,-2.8400584700013614,0.06564341941663877,Original +457,0.4067796610169492,0.3038841807909604,-4.6304395053630865,0.01897139383687084,Original +458,0.423728813559322,0.29950564971751414,-3.9160409861832215,0.02960312502298523,Original +459,0.4915254237288136,0.4095338983050847,-3.343711500585689,0.04426514489947514,Original +460,0.4,0.3516949152542373,-2.764906125414197,0.06986512747198882,Original +461,0.4745762711864407,0.3331920903954802,-3.755601850914824,0.03298931499378954,Original +462,0.4576271186440678,0.3418079096045198,-5.430582663966678,0.012254723173754265,Original +463,0.4067796610169492,0.342090395480226,-2.3719428909642666,0.09833105019693004,Original +464,0.5254237288135594,0.3714689265536723,-9.312498469112693,0.002621388573795643,Original +465,0.4067796610169492,0.3089453860640301,-4.904381946588596,0.03914970069711023,Original +466,0.5084745762711864,0.40120056497175144,-3.3115695002057812,0.04533888578886271,Original +467,0.423728813559322,0.34611581920903955,-5.009915149758147,0.015309062462291299,Original +468,0.4745762711864407,0.3670197740112994,-9.048660230550595,0.002850658340600586,Original +469,0.4406779661016949,0.34138418079096045,-2.360589363770645,0.09934685042336319,Original +470,0.4745762711864407,0.3801553672316384,-2.8398703872208553,0.06565358206650312,Original +471,0.4576271186440678,0.3372175141242938,-3.5534330922688557,0.037997540271468185,Original +472,0.5,0.44491525423728817,-2.26300952742407,0.10862358012074654,Original +473,0.4576271186440678,0.41779661016949154,-4.370123877984895,0.02216324865119666,Original +474,0.423728813559322,0.32033898305084746,-9.161458990088187,0.011705573839571451,Original +475,0.5254237288135594,0.384180790960452,-3.706246583305507,0.03413002778125795,Original +476,0.3898305084745763,0.3247881355932203,-4.8167110843371415,0.017046061853773893,Original +477,0.4745762711864407,0.35444915254237286,-3.7842912965837407,0.032348636596183485,Original +478,0.423728813559322,0.32492937853107345,-3.4140132990801026,0.04202684058034692,Original +479,0.5423728813559322,0.3884180790960452,-7.8459916831607694,0.004312165413432141,Original +480,0.3898305084745763,0.2825564971751412,-8.059631433861888,0.003989928963853392,Original +481,0.43333333333333335,0.3432203389830508,-4.253333333333336,0.023821477797343008,Original +482,0.4,0.3728813559322034,-3.9191835884530852,0.029541363360540956,Original +483,0.3898305084745763,0.3247175141242938,-3.2435043974379334,0.04772389993910521,Original +484,0.5333333333333333,0.385593220338983,-3.2052128901777346,0.04913559685870657,Original +485,0.5084745762711864,0.3458333333333333,-6.3744164409558755,0.007815387560659307,Original +486,0.559322033898305,0.3120056497175141,-10.204926179291313,0.0020055260919182947,Original +487,0.5423728813559322,0.3757062146892655,-2.741962532807021,0.07122087552292544,Original +488,0.5423728813559322,0.4221045197740113,-5.546054501123849,0.011558137961560508,Original +489,0.4576271186440678,0.37951977401129944,-4.250201094643769,0.02386812308620629,Original +490,0.45,0.3728813559322034,-4.984275273297009,0.015525883080703483,Original +491,0.5084745762711864,0.3332627118644068,-7.640503978788062,0.004655333124225627,Original +492,0.4745762711864407,0.37570621468926557,-6.286185570937117,0.008130066802486461,Original +493,0.3898305084745763,0.3500706214689266,-2.435222247729202,0.09289556214456558,Original +494,0.4576271186440678,0.4049435028248588,-5.1288742164266266,0.0143531474427786,Original +495,0.4666666666666667,0.3389830508474576,-18.452822728966602,0.00034730393994395734,Original +496,0.4406779661016949,0.39505649717514124,-3.8915662650602405,0.16012516703063404,Original +497,0.4915254237288136,0.38799435028248586,-4.361164169302447,0.022285069422793412,Original +498,0.4745762711864407,0.37973163841807916,-2.7216754734382707,0.07244696363099003,Original +499,0.4067796610169492,0.3500706214689266,-2.6495878836827105,0.07702097888753302,Original +500,0.3220338983050847,0.2699152542372881,-4.064774047804071,0.02685304092817379,Original +501,0.4406779661016949,0.354590395480226,-3.6417926698166023,0.035696879201596796,Original +502,0.4745762711864407,0.3714689265536723,-6.236810901332353,0.008313449992551876,Original +503,0.423728813559322,0.3586158192090395,-4.619015880036581,0.019098463373393156,Original +504,0.5084745762711864,0.39661016949152544,-2.9340578815309537,0.06080104286808068,Original +505,0.4067796610169492,0.31228813559322033,-6.309916959963368,0.008043815258589415,Original +506,0.4406779661016949,0.3627824858757063,-2.7319309661026785,0.07182390621092707,Original +507,0.5254237288135594,0.3716101694915254,-2.9643107979224634,0.059338263715170134,Original +508,0.4666666666666667,0.3601694915254237,-3.409705862611856,0.04215978877786793,Original +509,0.4915254237288136,0.3630649717514124,-5.7764663818628845,0.010315868020629015,Original +510,0.4576271186440678,0.33742937853107347,-6.023354344153489,0.009171855397206797,Original +511,0.5084745762711864,0.3258003766478343,-48.499999999999964,0.00042485397562817395,Original +512,0.4576271186440678,0.2867231638418079,-4.840000000000001,0.016823522712679524,Original +513,0.4406779661016949,0.3209745762711864,-3.5295270186824976,0.038652315352543654,Original +514,0.4067796610169492,0.3587570621468927,-2.6549539521063013,0.07666840199371955,Original +515,0.5254237288135594,0.286864406779661,-10.52321129742476,0.0018326710996097446,Original +516,0.5254237288135594,0.3799435028248587,-6.86666666666667,0.006324551982225255,Original +517,0.4745762711864407,0.3755649717514124,-3.418861432750501,0.04187784075273277,Original +518,0.4067796610169492,0.29964689265536726,-5.609714558498766,0.011196176065376182,Original +519,0.3389830508474576,0.32024482109227875,-10.47368421052631,0.008993145222343412,Original +520,0.4915254237288136,0.3706214689265537,-6.849973076409931,0.02065393933365802,Original +521,0.4,0.3601694915254237,-3.1333333333333346,0.05193110106234139,Original +522,0.45,0.3940677966101695,-5.245148133977578,0.013492692472278441,Original +523,0.4745762711864407,0.37175141242937854,-2.224480146264155,0.11257283375371135,Original +524,0.4576271186440678,0.33340395480225987,-4.859171643105984,0.016643155194105735,Original +525,0.45,0.4194915254237288,-2.8609898912604983,0.06452459428804105,Original +526,0.423728813559322,0.33333333333333337,-6.275716324421885,0.008168503479790335,Original +527,0.4406779661016949,0.29138418079096046,-5.949008298622876,0.009498040523452093,Original +528,0.4406779661016949,0.3882768361581921,-4.096759822564065,0.026305115270877533,Original +529,0.5,0.3389830508474576,-2.4666312377251187,0.09033399701898738,Original +530,0.4745762711864407,0.39216101694915256,-2.6138797970211702,0.07941891525657455,Original +531,0.3898305084745763,0.2865348399246704,-10.417426280656978,0.009089216213235031,Original +532,0.559322033898305,0.40494350282485875,-3.5632864433698304,0.03773178159693135,Original +533,0.4067796610169492,0.3081214689265537,-2.706180513915112,0.07340111077355108,Original +534,0.4745762711864407,0.3375706214689265,-3.298098944452524,0.04579869976521974,Original +535,0.423728813559322,0.30374293785310735,-3.6378487523667777,0.0357957127768318,Original +536,0.5254237288135594,0.37980225988700567,-5.144656775656592,0.014232232343107012,Original +537,0.4067796610169492,0.3334039548022599,-5.550438587280521,0.011532723915216122,Original +538,0.4576271186440678,0.3456214689265537,-2.619445228666292,0.07903917160851617,Original +539,0.4406779661016949,0.34597457627118644,-22.472055097835053,0.0001929545430821344,Original +540,0.3389830508474576,0.28707627118644075,-3.9963952715287125,0.02807461925885932,Original +541,0.48333333333333334,0.3855932203389831,-3.388738579570791,0.042814628734577545,Original +542,0.4915254237288136,0.4495291902071563,-3.306335305824179,0.08057456813744839,Original +543,0.4406779661016949,0.33326271186440676,-4.308228721563899,0.023022768426326834,Original +544,0.3898305084745763,0.30409604519774014,-3.009193969690081,0.057248412075778514,Original +545,0.4576271186440678,0.29526836158192094,-7.3968729657562395,0.005110511890500865,Original +546,0.4745762711864407,0.3627824858757062,-4.484419263456091,0.020682260794596535,Original +547,0.4666666666666667,0.3432203389830508,-8.529366637570185,0.0033856068172450396,Original +548,0.5254237288135594,0.3163135593220339,-7.643205738383732,0.004650591457090468,Original +549,0.4067796610169492,0.2828389830508474,-3.1258403580998007,0.052233941204207116,Original +550,0.4666666666666667,0.39406779661016955,-2.982530505374287,0.058478563965002554,Original +551,0.3559322033898305,0.30903954802259886,-7.0399681163437675,0.019586257511572443,Original +552,0.4406779661016949,0.38813559322033897,-1.8603721116372136,0.15977096644438923,Original +553,0.5254237288135594,0.358545197740113,-4.751834676331621,0.017686518806967016,Original +554,0.4915254237288136,0.3206920903954802,-3.269035331124903,0.046811083782206005,Original +555,0.4915254237288136,0.3500706214689266,-3.7595621188342796,0.032899914767886236,Original +556,0.4576271186440678,0.3628531073446328,-5.912181117649445,0.009665258648047835,Original +557,0.5084745762711864,0.38834745762711864,-3.4214826476365072,0.04179756150788246,Original +558,0.5,0.35169491525423724,-4.748264132284415,0.017722668444947104,Original +559,0.3728813559322034,0.274364406779661,-4.35035650844239,0.022433171283776563,Original +560,0.2542372881355932,0.19851694915254237,-3.3448722836047113,0.04422697438835633,Original +561,0.4915254237288136,0.41370056497175145,-3.4764271828366216,0.04015895679106413,Original +562,0.423728813559322,0.341454802259887,-2.2689902462968896,0.10802564058574593,Original +563,0.4745762711864407,0.37153954802259886,-3.9815118119797472,0.028349938061132722,Original +564,0.4067796610169492,0.3755649717514124,-6.27344916163044,0.008176858545156238,Original +565,0.4406779661016949,0.3840395480225989,-9.780487804878028,0.002271338003074173,Original +566,0.4406779661016949,0.35451977401129947,-4.066666666666665,0.02682021272712732,Original +567,0.423728813559322,0.4050141242937853,-3.35873852777973,0.0437742186193112,Original +568,0.4915254237288136,0.3879943502824859,-4.361164169302444,0.02228506942279346,Original +569,0.4406779661016949,0.3421610169491526,-3.2068965517241366,0.049072424069962796,Original +570,0.4915254237288136,0.405225988700565,-3.4882407793268584,0.03981737779020827,Original +571,0.576271186440678,0.4471751412429379,-6.645559382413252,0.006943215258999799,Original +572,0.576271186440678,0.4717514124293785,-6.08276253029822,0.02597846598858569,Original +573,0.4576271186440678,0.2995762711864407,-10.972170265475434,0.0016209009976600197,Original +574,0.3728813559322034,0.33145009416195853,-2.8168113585237156,0.10631108992737348,Original +575,0.4067796610169492,0.3713983050847458,-3.303424358920434,0.045616214777893696,Original +576,0.5084745762711864,0.3542372881355932,-9.208061462894172,0.0027091139233435414,Original +577,0.5084745762711864,0.35007062146892653,-8.932852690514894,0.0029596885792663864,Original +578,0.4745762711864407,0.3626412429378531,-3.365301828336874,0.04356196753364835,Original +579,0.3728813559322034,0.2785310734463277,-5.283646176355578,0.013222684385745894,Original +580,0.4166666666666667,0.3516949152542373,-4.489140335563251,0.02062388445422241,Original +581,0.5254237288135594,0.40098870056497177,-7.988978716899146,0.004092863743178219,Original +582,0.3728813559322034,0.28255649717514125,-6.786220279071329,0.0065409101032541635,Original +583,0.4406779661016949,0.33771186440677964,-5.3858337188310035,0.012539411668401262,Original +584,0.4067796610169492,0.38757062146892657,-2.4285714285714284,0.13584143478190688,Original +585,0.3898305084745763,0.3417372881355932,-6.45766024002191,0.007533010897429456,Original +586,0.4576271186440678,0.3964689265536723,-4.172537109479583,0.02506338811524026,Original +587,0.4067796610169492,0.3169491525423729,-1.9629629629629635,0.14443412236891323,Original +588,0.4666666666666667,0.364406779661017,-6.301611145596256,0.008073865317362929,Original +589,0.423728813559322,0.3463983050847458,-2.043874646199097,0.13354795806792266,Original +590,0.4406779661016949,0.3966101694915254,-3.050041359322525,0.05542620511793901,Original +591,0.4576271186440678,0.3632062146892655,-2.9222455629486523,0.06138443122939498,Original +592,0.4406779661016949,0.3459745762711864,-3.610360849034176,0.03649435588128962,Original +593,0.4067796610169492,0.33785310734463275,-2.653428192571171,0.07676844842248204,Original +594,0.4067796610169492,0.346045197740113,-3.3888747468281344,0.04281033448755565,Original +595,0.38333333333333336,0.3305084745762712,-2.6185237571323796,0.07910189090563483,Original +596,0.4406779661016949,0.3377118644067797,-6.269358747755155,0.00819196116993917,Original +597,0.4406779661016949,0.38389830508474576,-8.48622665150705,0.0034358339331581286,Original +598,0.559322033898305,0.4096045197740113,-4.528095585898831,0.020150257453059105,Original +599,0.423728813559322,0.3882062146892655,-4.997867054186407,0.015410451099079129,Original +600,0.2033898305084746,0.12245762711864408,-3.608916325424141,0.03653155025346661,Original +601,0.3050847457627119,0.24032485875706217,-3.0165616144994942,0.05691423813610457,Original +602,0.2711864406779661,0.20247175141242937,-11.096292845694657,0.00156813241982992,Original +603,0.22033898305084745,0.14759887005649716,-2.8959560191449962,0.06270817363169148,Original +604,0.1694915254237288,0.1265536723163842,-3.972583638518804,0.02851676236015958,Original +605,0.22033898305084745,0.13933615819209041,-4.2672442091017775,0.02361573435774843,Original +606,0.22033898305084745,0.15188323917137478,-3.3981981491560638,0.0767589601591142,Original +607,0.3333333333333333,0.2584745762711864,-5.172270386627225,0.014023853313803805,Original +608,0.23728813559322035,0.17711864406779662,-5.84274742881549,0.00999117759634526,Original +609,0.25,0.1694915254237288,-3.878358759406699,0.03035678137281933,Original +610,0.22033898305084745,0.13057909604519774,-4.374936505877199,0.02209816935791197,Original +611,0.2711864406779661,0.18128531073446327,-4.128224087608492,0.025780103759508587,Original +612,0.1694915254237288,0.10677966101694915,-2.53917501033781,0.12636332880012854,Original +613,0.2033898305084746,0.1518361581920904,-3.0820624157214693,0.054048493565592826,Original +614,0.15254237288135594,0.12648305084745762,-2.492276285625687,0.0883063687998794,Original +615,0.23728813559322035,0.21101694915254238,-4.841386618546792,0.016810392289180406,Original +616,0.2033898305084746,0.13465160075329566,-2.718662067942085,0.11285123312157036,Original +617,0.1864406779661017,0.1602401129943503,-2.531317735884927,0.08532554275225189,Original +618,0.288135593220339,0.19406779661016949,-8.748025509254019,0.0031454221399862072,Original +619,0.23728813559322035,0.15176553672316384,-5.734186911097393,0.01053023168597254,Original +620,0.3389830508474576,0.25296610169491524,-2.7041039897995947,0.07353015997518966,Original +621,0.2033898305084746,0.15190677966101696,-7.409133468514449,0.005086235812941152,Original +622,0.23333333333333334,0.15254237288135594,-3.5204166420089162,0.038905627181690466,Original +623,0.23728813559322035,0.1899717514124294,-5.4166289591447665,0.012342580506632655,Original +624,0.22033898305084745,0.1435734463276836,-5.128214329323895,0.01435823217533278,Original +625,0.2542372881355932,0.16864406779661018,-6.5059819266483885,0.00737522940630693,Original +626,0.23333333333333334,0.20338983050847456,-2.4984439601924695,0.08782706589330419,Original +627,0.23728813559322035,0.18559322033898307,-8.275546059124256,0.0036958123216244147,Original +628,0.5423728813559322,0.48107344632768356,-2.69383410385129,0.07417255068242208,Original +629,0.2542372881355932,0.1643361581920904,-5.023778124999512,0.015193473744699431,Original +630,0.15254237288135594,0.11233521657250471,-2.7098687752823145,0.11346466393886437,Original +631,0.3050847457627119,0.1689265536723164,-5.86064261618386,0.00990581703436056,Original +632,0.23728813559322035,0.15600282485875705,-3.1321496288779005,0.051978794780651254,Original +633,0.2542372881355932,0.17281073446327683,-2.714263880607111,0.07290142241990857,Original +634,0.22033898305084745,0.16878531073446326,-4.294117647058823,0.023224732577996746,Original +635,0.2,0.17372881355932202,-6.200000000000003,0.008453719117567243,Original +636,0.5423728813559322,0.3711158192090396,-5.424838731783427,0.01229078962720508,Original +637,0.288135593220339,0.18566384180790962,-4.932949247691678,0.01597207780336412,Original +638,0.26666666666666666,0.19491525423728812,-6.558251799577894,0.0072094124531609394,Original +639,0.1694915254237288,0.12662429378531073,-4.954648074701186,0.01578143114842512,Original +640,0.3559322033898305,0.18135593220338986,-7.923076923076922,0.004192062411449591,Original +641,0.23333333333333334,0.1398305084745763,-4.195254538472044,0.02470591794360766,Original +642,0.22033898305084745,0.1730225988700565,-4.389315939938801,0.021905189439865372,Original +643,0.2542372881355932,0.17718926553672315,-9.34117796358571,0.0025979582105049733,Original +644,0.22033898305084745,0.13483992467043315,-8.711460465745363,0.012922185254776281,Original +645,0.22033898305084745,0.1812853107344633,-2.856098645795084,0.06478389664989048,Original +646,0.1694915254237288,0.1266949152542373,-3.796706309892804,0.032076336335065445,Original +647,0.18333333333333332,0.13559322033898308,-3.983368200684214,0.02831540894214516,Original +648,0.23728813559322035,0.16913841807909605,-2.2349882150268345,0.11147884166525937,Original +649,0.2711864406779661,0.19858757062146892,-2.5535821728149255,0.08368093910992941,Original +650,0.423728813559322,0.39653954802259883,-2.821061725972151,0.06667985231722355,Original +651,0.2542372881355932,0.22365819209039547,-6.611650272701572,0.007045041561568846,Original +652,0.15254237288135594,0.1290960451977401,-1.6375555461118865,0.24316808629278763,Original +653,0.22033898305084745,0.19830508474576272,-5.461092327709232,0.01206545301607257,Original +654,0.3220338983050847,0.16927966101694913,-3.519195236968839,0.038939748753238246,Original +655,0.23728813559322035,0.1772598870056497,-6.871842709362766,0.00631095591061526,Original +656,0.288135593220339,0.21094632768361582,-3.6366923315065693,0.035824758686131074,Original +657,0.3559322033898305,0.2918079096045198,-2.173368590328213,0.16182504545842755,Original +658,0.2833333333333333,0.1610169491525424,-3.7266706420173588,0.033651949508593376,Original +659,0.3050847457627119,0.18524011299435028,-3.1980807339949875,0.04940434566953938,Original +660,0.2711864406779661,0.1602401129943503,-4.406927169154064,0.02167179821796596,Original +661,0.2542372881355932,0.16468926553672317,-8.002907374821234,0.004072295634170163,Original +662,0.2033898305084746,0.13919491525423727,-8.229362053239296,0.003756251434707851,Original +663,0.5333333333333333,0.44067796610169496,-2.3309944988213394,0.10205524591620584,Original +664,0.23728813559322035,0.16899717514124293,-3.0669989094316277,0.054691170931315904,Original +665,0.21666666666666667,0.15677966101694918,-2.8266666666666658,0.06637194694612157,Original +666,0.22033898305084745,0.17740112994350282,-2.5678020379180455,0.0826509420493295,Original +667,0.2711864406779661,0.19399717514124293,-7.533632446839843,0.00484812769882256,Original +668,0.21666666666666667,0.1228813559322034,-4.990929625938783,0.015469228914864434,Original +669,0.423728813559322,0.4007768361581921,-2.3943057732222077,0.09636694644723628,Original +670,0.3389830508474576,0.3036723163841808,-2.439750182371333,0.09252085884337388,Original +671,0.1864406779661017,0.14058380414312618,-2.9676921508875798,0.0972607614111294,Original +672,0.2542372881355932,0.18997175141242936,-2.727682454452322,0.07208120831037232,Original +673,0.22033898305084745,0.17323446327683617,-2.443879350950321,0.09218076354483482,Original +674,0.2542372881355932,0.1606638418079096,-2.8273201672963535,0.06633616259613716,Original +675,0.2542372881355932,0.16885593220338985,-6.94311791030617,0.00612758913066015,Original +676,0.2833333333333333,0.1694915254237288,-8.226203052846838,0.003760433078739023,Original +677,0.2542372881355932,0.21511299435028247,-3.898316388834917,0.029954591830056016,Original +678,0.26666666666666666,0.15677966101694915,-4.3423725516066725,0.022543397382713576,Original +679,0.2,0.14830508474576273,-12.19999999999999,0.001185729108116372,Original +680,0.23728813559322035,0.12245762711864408,-8.865808969091209,0.0030253402213844243,Original +681,0.21666666666666667,0.15254237288135594,-4.144434018455757,0.025514895723579813,Original +682,0.2711864406779661,0.1899011299435028,-7.530600502193264,0.004853749660576098,Original +683,0.13333333333333333,0.1016949152542373,-3.2331615074619027,0.048100099388329834,Original +684,0.2711864406779661,0.1899717514124294,-3.1834100610600315,0.04996302047691425,Original +685,0.23728813559322035,0.18566384180790962,-4.298438567165345,0.023162647133957697,Original +686,0.18333333333333332,0.15677966101694918,-6.266666666666664,0.008201920869688295,Original +687,0.2711864406779661,0.2027542372881356,-3.621297736079414,0.0362143108175595,Original +688,0.1864406779661017,0.16454802259887005,-5.39640733462664,0.012471369780280921,Original +689,0.1864406779661017,0.11807909604519774,-10.57181908559856,0.0018080338565436176,Original +690,0.5254237288135594,0.3463276836158192,-6.147522027439089,0.008659111206240855,Original +691,0.3050847457627119,0.2615819209039548,-4.063904567563298,0.026868139751298294,Original +692,0.22033898305084745,0.1644774011299435,-5.495043416350755,0.011859301638986507,Original +693,0.5423728813559322,0.45593220338983054,-4.34666107141259,0.02248410342384351,Original +694,0.23728813559322035,0.2068502824858757,-3.4595325465113227,0.04065396019819478,Original +695,0.3389830508474576,0.17711864406779662,-8.161598246688756,0.0038473182454779415,Original +696,0.2711864406779661,0.1730225988700565,-6.742490376010055,0.006662617945926082,Original +697,0.288135593220339,0.15176553672316384,-5.775098949219386,0.010322711204428422,Original +698,0.2033898305084746,0.1434322033898305,-13.161604379732514,0.0009475271511557978,Original +699,0.23728813559322035,0.18121468926553674,-2.37554254984018,0.0980116289989706,Original +700,0.21666666666666667,0.1440677966101695,-4.11529355279074,0.025994211990808435,Original +701,0.2711864406779661,0.19837570621468928,-3.4094802718971997,0.042166766315337265,Original +702,0.288135593220339,0.1731638418079096,-7.679245283018867,0.004587946327225194,Original +703,0.21666666666666667,0.1440677966101695,-4.945967306057795,0.015857343159367175,Original +704,0.4067796610169492,0.3418079096045198,-5.938574464184704,0.00954502881900154,Original +705,0.1864406779661017,0.13509887005649718,-4.199506762976739,0.02463973970623848,Original +706,0.1864406779661017,0.1346045197740113,-2.8897637795275584,0.21209033041734834,Original +707,0.2033898305084746,0.14343220338983054,-4.115302637249344,0.025994060763921405,Original +708,0.5932203389830508,0.4980225988700565,-2.5424731748068985,0.08449660749382112,Original +709,0.2711864406779661,0.17747175141242938,-4.301532117285934,0.023118328990842697,Original +710,0.3333333333333333,0.2076271186440678,-3.6796971261465163,0.03476456233229096,Original +711,0.23728813559322035,0.17309322033898306,-2.2132516655917454,0.11375613879223859,Original +712,0.2542372881355932,0.17323446327683617,-3.2772900040790653,0.046520687780565555,Original +713,0.2542372881355932,0.18149717514124292,-14.714285714285706,0.0006808928260169268,Original +714,0.5254237288135594,0.417725988700565,-10.18365359023622,0.0020178363490044415,Original +715,0.5254237288135594,0.4609227871939736,-2.429062741718821,0.13579717414652953,Original +716,0.2711864406779661,0.2024717514124294,-4.532253548113998,0.020100540272823354,Original +717,0.23728813559322035,0.14774011299435028,-3.52461629327718,0.03878859478065401,Original +718,0.23333333333333334,0.16101694915254236,-6.609891577527327,0.007050376214881156,Original +719,0.2711864406779661,0.21525423728813559,-6.513721780101715,0.007350361959320121,Original +720,0.5254237288135594,0.4175847457627119,-8.4047123277365,0.0035334622322826825,Original +721,0.3,0.1652542372881356,-6.045729350003519,0.009076571184230395,Original +722,0.23728813559322035,0.16440677966101697,-2.6381526558147996,0.07777904054371029,Original +723,0.3,0.19491525423728814,-3.201666232864797,0.04926900790572485,Original +724,0.1864406779661017,0.15176553672316384,-3.080747359192189,0.05410421984381787,Original +725,0.3220338983050847,0.19823446327683616,-7.231156579885582,0.005454152940167432,Original +726,0.26666666666666666,0.18220338983050846,-4.8345434590575715,0.016875322539984233,Original +727,0.2542372881355932,0.15621468926553672,-6.014126755966144,0.009211533394928783,Original +728,0.5254237288135594,0.4514124293785311,-2.773457470011942,0.06936804083315011,Original +729,0.2711864406779661,0.1942090395480226,-6.695816773490067,0.006795828785145915,Original +730,0.2542372881355932,0.19823446327683614,-4.580754969794434,0.019532183920823603,Original +731,0.2,0.11440677966101695,-4.899219625733926,0.016274424287796678,Original +732,0.3220338983050847,0.21504237288135594,-2.93071839321159,0.060965267594478456,Original +733,0.2542372881355932,0.1772598870056497,-4.332361052668546,0.022682607763125775,Original +734,0.2033898305084746,0.1348870056497175,-2.4242425425396985,0.09381189733415801,Original +735,0.23728813559322035,0.17747175141242938,-2.3175577935580045,0.10331445267128257,Original +736,0.5932203389830508,0.5064971751412429,-3.4740867918071037,0.04022706931462362,Original +737,0.35,0.2542372881355932,-5.230883563714916,0.013594564408150057,Original +738,0.15254237288135594,0.11377118644067798,-2.1108984192580493,0.12526011393630182,Original +739,0.2033898305084746,0.16031073446327682,-2.958934625443265,0.05959496241150818,Original +740,0.2542372881355932,0.1480225988700565,-3.1535457618776688,0.05112515625755018,Original +741,0.288135593220339,0.1812853107344633,-7.814244577012594,0.004362950284160135,Original +742,0.288135593220339,0.12655367231638417,-9.201899164855353,0.0027144108276950137,Original +743,0.2542372881355932,0.19830508474576272,-5.284229075567873,0.01321865065791954,Original +744,0.3559322033898305,0.1475988700564972,-7.72844712666813,0.00450420801537689,Original +745,0.18333333333333332,0.1440677966101695,-3.5889645674855384,0.037050266207125974,Original +746,0.22033898305084745,0.13919491525423727,-4.348997984611037,0.022451877691097657,Original +747,0.23333333333333334,0.2076271186440678,-6.0666666666666735,0.008988582402196459,Original +748,0.23728813559322035,0.16857344632768362,-3.433022580597172,0.04144644945716894,Original +749,0.21666666666666667,0.1694915254237288,-2.5768659443669355,0.08200254809923566,Original +750,0.22033898305084745,0.12217514124293785,-6.391226809216671,0.0077572568493554745,Original +751,0.2711864406779661,0.15204802259887007,-4.146966401618811,0.025473782147724618,Original +752,0.23728813559322035,0.1856638418079096,-7.414159169988326,0.005076328764679312,Original +753,0.3389830508474576,0.2827683615819209,-2.946612591077136,0.06018857902224991,Original +754,0.23728813559322035,0.16857344632768362,-3.9354028393368354,0.02922522554345655,Original +755,0.288135593220339,0.1855225988700565,-9.224885508045242,0.002694722054185784,Original +756,0.2542372881355932,0.18149717514124294,-5.610767520239283,0.011190314670533465,Original +757,0.3050847457627119,0.21532485875706214,-4.672257354206355,0.018515494454469265,Original +758,0.2711864406779661,0.19413841807909604,-14.324297025025746,0.000737365952799921,Original +759,0.15,0.1271186440677966,-4.67653718043597,0.018469641646674036,Original +760,0.2542372881355932,0.23022598870056496,-2.4285714285714284,0.13584143478190688,Original +761,0.2542372881355932,0.1571563088512241,-6.947685432148197,0.020094339591082675,Original +762,0.2033898305084746,0.12919020715630886,-5.005419915905,0.03767240357329794,Original +763,0.1694915254237288,0.13072033898305085,-2.4581295560193186,0.09101873468865801,Original +764,0.5084745762711864,0.4389830508474576,-4.148687032732543,0.025445896223527123,Original +765,0.2033898305084746,0.14745762711864407,-3.1135770454260725,0.052734401788326495,Original +766,0.1864406779661017,0.1432909604519774,-3.097057119520469,0.053418165120319126,Original +767,0.23728813559322035,0.15197740112994348,-5.438132165795431,0.012207529595140413,Original +768,0.22033898305084745,0.16871468926553673,-5.51623494926466,0.011732957333623041,Original +769,0.23333333333333334,0.15677966101694915,-2.5809523809523816,0.08171227084668078,Original +770,0.23728813559322035,0.1393361581920904,-11.45967949880383,0.001426179984263791,Original +771,0.21666666666666667,0.13983050847457626,-3.156608161329988,0.05100442714599591,Original +772,0.23728813559322035,0.20268361581920905,-2.734913303711143,0.07164396808065115,Original +773,0.22033898305084745,0.16318267419962337,-2.2651619180423666,0.15174721013539427,Original +774,0.2542372881355932,0.1814265536723164,-3.4427734423525815,0.04115269085326815,Original +775,0.2033898305084746,0.16031073446327682,-5.211581712072246,0.01373401507365159,Original +776,0.3559322033898305,0.2573446327683616,-5.309251443565037,0.01304699781574515,Original +777,0.3898305084745763,0.33709981167608283,-28.00000000000001,0.0012730749910096228,Original +778,0.3220338983050847,0.14759887005649716,-7.540447009277395,0.00483552316841547,Original +779,0.22033898305084745,0.1561440677966102,-14.299454098945166,0.0007411711769066464,Original +780,0.22033898305084745,0.13481638418079095,-4.017937620623259,0.027682186108019237,Original +781,0.23728813559322035,0.1426553672316384,-3.941176470588237,0.15819200517512308,Original +782,0.15254237288135594,0.10988700564971751,-2.589247891896882,0.08112688802619472,Original +783,0.21666666666666667,0.13559322033898305,-4.428506143912038,0.021390194677135307,Original +784,0.22033898305084745,0.16864406779661018,-3.4776799529026854,0.040122557696007824,Original +785,0.2711864406779661,0.17309322033898306,-7.501973136790743,0.004907258242497694,Original +786,0.3728813559322034,0.24449152542372882,-3.749896613676689,0.03311865459697996,Original +787,0.23333333333333334,0.1271186440677966,-7.236123373843309,0.005443417574107074,Original +788,0.1864406779661017,0.14613935969868172,-3.446237951244447,0.07486648577768765,Original +789,0.2542372881355932,0.16871468926553673,-4.7295667427478545,0.017913544696107814,Original +790,0.5254237288135594,0.4600988700564972,-2.332818394140734,0.10188575953545359,Original +791,0.22033898305084745,0.1814265536723164,-4.854861096609305,0.016683488889815655,Original +792,0.2542372881355932,0.1350282485875706,-5.941894164731409,0.027174489715421215,Original +793,0.2542372881355932,0.21518361581920906,-2.4427691887863823,0.09227205070358749,Original +794,0.38333333333333336,0.21610169491525422,-9.572072668100612,0.002419061759112068,Original +795,0.2711864406779661,0.19823446327683616,-5.967112085495144,0.009417228305284209,Original +796,0.15254237288135594,0.12238700564971752,-6.812191696797892,0.00647001003369456,Original +797,0.3898305084745763,0.22379943502824862,-6.706721318121227,0.006764394593913462,Original +798,0.288135593220339,0.17281073446327685,-5.072284754402177,0.014797870994727638,Original +799,0.1864406779661017,0.16299435028248588,-3.608695652173914,0.06894253641177729,Original +800,0.5254237288135594,0.38411016949152543,-7.205444253314168,0.005510176845082696,Original +801,0.4576271186440678,0.36563088512241054,-2.235697940503432,0.1548857125971983,Original +802,0.45,0.36864406779661013,-3.21491849579106,0.04877283804441102,Original +803,0.5254237288135594,0.4011299435028249,-4.050528318752622,0.027101807358158075,Original +804,0.4745762711864407,0.36737288135593216,-3.6999746626775307,0.03427858129419643,Original +805,0.4406779661016949,0.4135593220338983,-4.495611895592143,0.02054420972468061,Original +806,0.3728813559322034,0.3290960451977401,-3.025290226140455,0.05652149905978702,Original +807,0.5084745762711864,0.4639830508474576,-2.3727172680373334,0.09826222788434269,Original +808,0.4915254237288136,0.43898305084745765,-3.136812146700215,0.051791252064485205,Original +809,0.4166666666666667,0.35169491525423724,-2.252629691622204,0.10967084489225898,Original +810,0.4576271186440678,0.4091101694915254,-2.4465756114105797,0.09195951277963867,Original +811,0.4576271186440678,0.4260593220338983,-2.3835678188830465,0.0973040279791015,Original +812,0.3728813559322034,0.32902542372881355,-6.272727272727274,0.0081795212436495,Original +813,0.4406779661016949,0.3584039548022599,-2.8942424583125095,0.06279568913296162,Original +814,0.5423728813559322,0.3962570621468927,-4.31678224360102,0.022901451094550856,Original +815,0.5084745762711864,0.438771186440678,-3.9134326914916078,0.029654511991928277,Original +816,0.4406779661016949,0.3586158192090395,-11.057143071059498,0.0015845269383183932,Original +817,0.3728813559322034,0.3093220338983051,-2.0349703424076058,0.1788266478229615,Original +818,0.5254237288135594,0.4639124293785311,-3.1846701192429343,0.049914723854652635,Original +819,0.4576271186440678,0.40091807909604515,-2.9501131862499728,0.060019186645291704,Original +820,0.4745762711864407,0.41572504708097924,-2.4051807590126426,0.13797203364885854,Original +821,0.4067796610169492,0.3145009416195857,-9.800000000000015,0.010252475022698292,Original +822,0.5254237288135594,0.42664783427495295,-3.434253416143983,0.07533225481738264,Original +823,0.423728813559322,0.35847457627118645,-3.7468218576747336,0.03318862977572546,Original +824,0.48333333333333334,0.4152542372881356,-2.677777777777778,0.07519088494134536,Original +825,0.423728813559322,0.3415960451977401,-2.404160372045395,0.09551661130031602,Original +826,0.4666666666666667,0.3686440677966102,-3.7363704002836675,0.03342790419303777,Original +827,0.45,0.3855932203389831,-3.427505887692899,0.041613829518252106,Original +828,0.5254237288135594,0.45572033898305087,-2.7857691081092364,0.06866006993419116,Original +829,0.5333333333333333,0.4322033898305085,-5.013002700820275,0.015283219713031168,Original +830,0.5084745762711864,0.41786723163841805,-2.9961510064713686,0.057846064903379533,Original +831,0.576271186440678,0.384180790960452,-4.605541138054455,0.01924977204112997,Original +832,0.4745762711864407,0.4048022598870057,-3.1550201885686455,0.051066984905793825,Original +833,0.4576271186440678,0.3936911487758945,-1.6469089297144321,0.24133036369239244,Original +834,0.48333333333333334,0.38983050847457623,-6.0432055511403355,0.009087253623111777,Original +835,0.4067796610169492,0.3036723163841808,-4.41816098848226,0.021524602925384974,Original +836,0.5084745762711864,0.4557203389830508,-7.350069795231334,0.00520460105096178,Original +837,0.4406779661016949,0.3461864406779661,-5.940860662234962,0.009534706994175631,Original +838,0.4915254237288136,0.3879237288135593,-3.2460121747166957,0.04763324643436874,Original +839,0.5084745762711864,0.3924435028248588,-4.124951949583921,0.02583406935919925,Original +840,0.4166666666666667,0.38135593220338987,-1.7503501050350099,0.17836241760761135,Original +841,0.4915254237288136,0.396680790960452,-3.34946047620416,0.04407650716830172,Original +842,0.5,0.4406779661016949,-3.8340579025361636,0.03127462797536876,Original +843,0.4576271186440678,0.3418079096045198,-5.165514464459437,0.014074465381170985,Original +844,0.5084745762711864,0.3840395480225988,-6.480569535567828,0.007457659131028016,Original +845,0.5166666666666667,0.4491525423728813,-6.170953464957155,0.008566603032878853,Original +846,0.423728813559322,0.33728813559322035,-2.5791808608632842,0.08183795390499486,Original +847,0.4745762711864407,0.3992467043314501,-2.0879830038875222,0.17203960039889202,Original +848,0.559322033898305,0.4053672316384181,-5.498322668127967,0.01183963469227656,Original +849,0.4576271186440678,0.36292372881355933,-6.299107729795289,0.008082951481625023,Original +850,0.45,0.3771186440677966,-2.0909375180606187,0.12766278225478178,Original +851,0.4067796610169492,0.3423728813559322,-1.7039616353344613,0.2305022123029592,Original +852,0.4576271186440678,0.32902542372881355,-6.562385865517576,0.007196507966961928,Original +853,0.4406779661016949,0.3251412429378531,-5.210719601593815,0.013740287056139356,Original +854,0.4067796610169492,0.32627118644067793,-2.0939473213563398,0.17129801016782423,Original +855,0.4067796610169492,0.32937853107344633,-3.048964780184798,0.05547328824097628,Original +856,0.5084745762711864,0.39661016949152544,-7.7424126813571705,0.004480807680029333,Original +857,0.4576271186440678,0.37608286252354045,-1.5525897646537226,0.2607163805465733,Original +858,0.423728813559322,0.3336864406779661,-3.1952478845632513,0.04951160636548708,Original +859,0.4745762711864407,0.3458333333333333,-9.810579653543469,0.002251005507538249,Original +860,0.4745762711864407,0.36257062146892655,-4.233384268372494,0.0241205868939632,Original +861,0.4406779661016949,0.4009180790960452,-4.384410769545493,0.02197077351423448,Original +862,0.4,0.3220338983050847,-5.039047529047533,0.015067470993325742,Original +863,0.4576271186440678,0.37959039548022605,-3.066783460464022,0.05470043251723667,Original +864,0.5,0.42372881355932207,-2.84604989415154,0.06532071006198013,Original +865,0.4576271186440678,0.3458333333333333,-3.64966532468076,0.03550063326417129,Original +866,0.5084745762711864,0.358545197740113,-3.7292186611760196,0.033592909525737194,Original +867,0.4576271186440678,0.3461864406779661,-3.667546560934897,0.03505999613164613,Original +868,0.4406779661016949,0.34173728813559323,-3.266799462776711,0.046890135550055285,Original +869,0.5423728813559322,0.40508474576271186,-5.713299488454646,0.010638289400887548,Original +870,0.4067796610169492,0.2995056497175141,-4.670840095116222,0.018530711133626115,Original +871,0.4166666666666667,0.3644067796610169,-3.5603266600026933,0.03781136121703687,Original +872,0.3389830508474576,0.295409604519774,-3.8525647377658387,0.030886920710077346,Original +873,0.4915254237288136,0.3878531073446328,-2.7019645629286955,0.07366341268340729,Original +874,0.43333333333333335,0.3516949152542373,-2.830478177733986,0.06616357500554147,Original +875,0.36666666666666664,0.32627118644067793,-2.1497076400880912,0.12074123791130136,Original +876,0.4406779661016949,0.36285310734463283,-3.674885279264579,0.03488117727992117,Original +877,0.423728813559322,0.27415254237288134,-8.239062446111248,0.0037434491451463214,Original +878,0.4915254237288136,0.32916666666666666,-28.126664663270653,9.866049174120721e-05,Original +879,0.43333333333333335,0.3347457627118644,-2.188743887105837,0.11639121058222453,Original +880,0.5423728813559322,0.4218220338983051,-11.810674379463547,0.0013048196771810819,Original +881,0.3559322033898305,0.3093220338983051,-1.5714285714285712,0.25668888376056537,Original +882,0.5254237288135594,0.47662429378531074,-2.224165710326518,0.11260576823386355,Original +883,0.559322033898305,0.396680790960452,-4.238952655416746,0.024036610732601344,Original +884,0.3728813559322034,0.3206920903954802,-3.753591024171325,0.03303482712591477,Original +885,0.5254237288135594,0.40946327683615824,-3.475538999555497,0.04018478833317944,Original +886,0.4576271186440678,0.379590395480226,-3.6545153763584652,0.03538041968158101,Original +887,0.4915254237288136,0.40494350282485875,-2.9327458060449008,0.06086550016185654,Original +888,0.4915254237288136,0.43898305084745765,-2.551058736525074,0.08386536725088448,Original +889,0.4,0.35593220338983056,-2.84815729902686,0.06520767196759084,Original +890,0.4745762711864407,0.31645480225988704,-39.57411910381416,3.5500916860886294e-05,Original +891,0.5254237288135594,0.4260593220338983,-8.799079181288127,0.0030926117525078255,Original +892,0.4406779661016949,0.3880649717514124,-2.6501717516402645,0.07698251884827165,Original +893,0.4915254237288136,0.41793785310734466,-3.015997915032065,0.05693971913913344,Original +894,0.45,0.37711864406779666,-3.8784935044945925,0.030354042786245917,Original +895,0.4745762711864407,0.41619585687382293,-1.4090909090909103,0.29417736761437996,Original +896,0.5,0.39830508474576276,-9.295160030897799,0.0026356894602544566,Original +897,0.38333333333333336,0.3305084745762712,-3.5988166779486694,0.03679295873270534,Original +898,0.4166666666666667,0.3644067796610169,-1.628834176702405,0.20183517132355724,Original +899,0.5084745762711864,0.3845338983050847,-2.6313736841918445,0.07823278504898863,Original +900,0.559322033898305,0.4138418079096045,-3.773628023053654,0.032584885614146145,Original +901,0.4576271186440678,0.3712806026365348,-1.8625711948272987,0.20356174355735687,Original +902,0.4915254237288136,0.409180790960452,-4.9902021373122745,0.015475409518374506,Original +903,0.4576271186440678,0.3536723163841808,-3.085580966165473,0.09093394434924655,Original +904,0.4406779661016949,0.3163841807909604,-10.370899457402706,0.0019128148288002538,Original +905,0.4915254237288136,0.40070621468926554,-5.159875560151832,0.014116891909403921,Original +906,0.4067796610169492,0.33319209039548026,-4.493884775113415,0.02056543411778116,Original +907,0.5166666666666667,0.4152542372881356,-2.847055192728138,0.0652667568561372,Original +908,0.3898305084745763,0.33145009416195853,-11.716898663286049,0.007205446408459767,Original +909,0.3559322033898305,0.29936440677966103,-1.9119123291898108,0.1518400483442069,Original +910,0.3728813559322034,0.32005649717514123,-2.143768150852684,0.1652700646335296,Original +911,0.3220338983050847,0.2693973634651601,-2.180850112451567,0.1609698801291486,Original +912,0.4745762711864407,0.4132768361581921,-2.677380542667445,0.07521629738306508,Original +913,0.5333333333333333,0.3771186440677966,-7.008990210800726,0.005964312251936928,Original +914,0.576271186440678,0.400635593220339,-6.348003491637961,0.007907886129610865,Original +915,0.4915254237288136,0.3949152542372881,-18.999999999999993,0.03347541671314822,Original +916,0.4166666666666667,0.3474576271186441,-3.086709862908689,0.05385213329290739,Original +917,0.4915254237288136,0.3258003766478343,-15.805266578356209,0.003979221225814003,Original +918,0.3898305084745763,0.3459039548022599,-2.838910304269998,0.06570548867709089,Original +919,0.5254237288135594,0.4134887005649718,-3.585413143844187,0.037143581857500066,Original +920,0.5254237288135594,0.43848870056497175,-3.2975199061269667,0.04581859729168717,Original +921,0.4745762711864407,0.4009180790960452,-4.901036565932749,0.016257948615872354,Original +922,0.4576271186440678,0.3584039548022599,-3.1383427887418938,0.051729870414491945,Original +923,0.559322033898305,0.5021892655367232,-6.111799193237621,0.008802686470096136,Original +924,0.4745762711864407,0.3416666666666667,-6.884772011368271,0.00627716185131719,Original +925,0.423728813559322,0.3459039548022599,-2.5149332127933626,0.08656123080121365,Original +926,0.4745762711864407,0.39668079096045206,-3.806413998599051,0.03186546090943339,Original +927,0.4576271186440678,0.34187853107344635,-4.207255276582219,0.02451973533970688,Original +928,0.5084745762711864,0.3376412429378531,-4.4091197996331335,0.021642966336736617,Original +929,0.45,0.3728813559322034,-2.2290356659326926,0.11209698013636105,Original +930,0.4166666666666667,0.3474576271186441,-3.0867098629086884,0.05385213329290739,Original +931,0.45,0.3728813559322034,-3.7150594432211537,0.033922680348442695,Original +932,0.4576271186440678,0.3629943502824859,-3.5263157894736845,0.03874136219588886,Original +933,0.4745762711864407,0.3627824858757063,-5.7192755125875685,0.010607225660250866,Original +934,0.45,0.2923728813559322,-10.891044814105634,0.001656669068269899,Original +935,0.3898305084745763,0.34173728813559323,-17.461538461538474,0.0004093729804445589,Original +936,0.5166666666666667,0.4279661016949152,-4.429570467592002,0.021376428303786093,Original +937,0.3898305084745763,0.33319209039548026,-1.7611959878594554,0.17642304434957817,Original +938,0.4166666666666667,0.2966101694915254,-4.147575310031268,0.025463909186015793,Original +939,0.5084745762711864,0.4387005649717514,-6.879715746378599,0.006290349260396344,Original +940,0.4915254237288136,0.3545197740112994,-10.777777777777775,0.0017083811519259897,Original +941,0.4067796610169492,0.32895480225988705,-4.070774182545795,0.026749143973066495,Original +942,0.48333333333333334,0.40254237288135597,-2.414985892866683,0.09459301523161902,Original +943,0.4915254237288136,0.4221751412429378,-3.7480916030534375,0.03315971002355427,Original +944,0.4576271186440678,0.37139830508474575,-6.762389833500976,0.006606867955287357,Original +945,0.4576271186440678,0.33742937853107347,-4.058782494615296,0.02695730866033387,Original +946,0.4067796610169492,0.3629943502824859,-2.8181818181818175,0.06683875138704513,Original +947,0.5333333333333333,0.4152542372881356,-2.558118456493597,0.08335065260117518,Original +948,0.4067796610169492,0.2911016949152542,-2.67070108252106,0.07564520129729006,Original +949,0.5254237288135594,0.37139830508474575,-5.336986813075771,0.012860119949256482,Original +950,0.3220338983050847,0.2701271186440678,-4.975196209154729,0.015603617644457647,Original +951,0.5254237288135594,0.4682909604519774,-4.883695241304617,0.01641608650487274,Original +952,0.4576271186440678,0.34985875706214686,-2.370422697154792,0.09846632715247584,Original +953,0.5254237288135594,0.39272598870056497,-3.3006539570698283,0.04571103238074162,Original +954,0.55,0.4322033898305085,-4.815101245041479,0.017061585926028165,Original +955,0.559322033898305,0.4262005649717514,-4.051889951012785,0.027077901844717624,Original +956,0.4166666666666667,0.35169491525423724,-2.9151315524126016,0.06173916082876997,Original +957,0.4915254237288136,0.3755649717514124,-10.56303549746513,0.0018124532293534081,Original +958,0.423728813559322,0.2911016949152542,-10.742132190804174,0.0017250974273507926,Original +959,0.4666666666666667,0.3686440677966102,-2.812229917701684,0.06716864584643882,Original +960,0.4915254237288136,0.37570621468926557,-5.0089472186085136,0.015317175750995964,Original +961,0.4915254237288136,0.3753531073446328,-3.3382526588579973,0.044445211846365185,Original +962,0.4745762711864407,0.4302966101694915,-2.890628912501168,0.06298074227793468,Original +963,0.45,0.39830508474576276,-3.185621103989269,0.049878312712040275,Original +964,0.5084745762711864,0.35868644067796607,-5.327850954897623,0.012921287882286202,Original +965,0.4406779661016949,0.31207627118644066,-3.533152576889349,0.03855209262090739,Original +966,0.4406779661016949,0.3608757062146893,-1.4303797468354427,0.38842195960522763,Original +967,0.4067796610169492,0.29555084745762716,-7.084467611285733,0.005784199991243675,Original +968,0.4745762711864407,0.3987758945386064,-5.717277635038901,0.029257011705986744,Original +969,0.4067796610169492,0.32888418079096043,-2.287766368648155,0.10617410540553696,Original +970,0.4745762711864407,0.42627118644067796,-3.519233773599045,0.03893867159865643,Original +971,0.4915254237288136,0.3332627118644068,-7.238793509124532,0.005437657782298041,Original +972,0.5254237288135594,0.4092514124293785,-6.2886655641721925,0.008120996607005396,Original +973,0.48333333333333334,0.4025423728813559,-2.8010960513215215,0.06779122416384425,Original +974,0.5084745762711864,0.40473163841807913,-3.46978940590354,0.040352519825929224,Original +975,0.4576271186440678,0.3502824858757062,-4.196397990844169,0.024688099628686736,Original +976,0.4745762711864407,0.417725988700565,-7.10832364958241,0.005728765340993184,Original +977,0.5084745762711864,0.46836158192090394,-9.941988596598865,0.0021649872660866308,Original +978,0.4745762711864407,0.2995762711864407,-10.04928647585347,0.00209795503470734,Original +979,0.45,0.3220338983050848,-3.6987295116025987,0.03430817170101989,Original +980,0.3898305084745763,0.33312146892655364,-2.347903032601037,0.1004970270572768,Original +981,0.5254237288135594,0.3836864406779661,-3.485462485693465,0.03989737572645941,Original +982,0.4576271186440678,0.3925141242937853,-3.5865984774856337,0.03711240322472348,Original +983,0.36666666666666664,0.326271186440678,-3.788162541206022,0.03226341112647686,Original +984,0.5423728813559322,0.3923728813559322,-20.347964212104664,0.00025950431091567957,Original +985,0.4915254237288136,0.4134887005649717,-3.901282639398381,0.029895400966674732,Original +986,0.5166666666666667,0.3983050847457627,-5.867173552077365,0.009874902457062155,Original +987,0.38333333333333336,0.3220338983050848,-2.4570411640937726,0.09110685313385442,Original +988,0.45,0.40254237288135597,-3.2790242451070704,0.04645996816866966,Original +989,0.559322033898305,0.38418079096045193,-4.595745763298827,0.019360742557531358,Original +990,0.48333333333333334,0.3940677966101695,-4.750402896977879,0.017701003201901513,Original +991,0.5084745762711864,0.363135593220339,-4.5211227964641365,0.02023399027426734,Original +992,0.4915254237288136,0.4175141242937853,-2.8647998103065557,0.06432351407478958,Original +993,0.5084745762711864,0.3798022598870056,-14.122758056462086,0.0007690037706351588,Original +994,0.423728813559322,0.33764124293785314,-3.4045365361503497,0.042320045665658175,Original +995,0.4067796610169492,0.3712570621468927,-6.263567998756802,0.008213404490252622,Original +996,0.5254237288135594,0.4639830508474576,-3.2766095606229877,0.04654453906356892,Original +997,0.3728813559322034,0.3123587570621469,-5.08813713983205,0.01467150017877107,Original +998,0.48333333333333334,0.3728813559322034,-3.5693253330753327,0.03757007584643612,Original +999,0.45,0.364406779661017,-5.27455297545764,0.01328581900289993,Original +1000,0.4406779661016949,0.35451977401129947,-6.7777777777777715,0.006564177674076621,Original +1001,0.4,0.27118644067796605,-2.510197384488104,0.0869224831963583,Original +1002,0.4067796610169492,0.30338983050847457,-5.263055373168918,0.034257089055649254,Original +1003,0.3728813559322034,0.3077683615819209,-1.8092905922289133,0.16810978065452678,Original +1004,0.423728813559322,0.33742937853107347,-2.129023252677582,0.12312500629142373,Original +1005,0.3559322033898305,0.3163841807909604,-4.041451884327386,0.02726185440936123,Original +1006,0.5254237288135594,0.4301553672316384,-2.476997933163102,0.08950755799392057,Original +1007,0.43333333333333335,0.3305084745762712,-3.9024896268095732,0.029871358819325514,Original +1008,0.423728813559322,0.36687853107344637,-2.115310434853906,0.12473633813156884,Original +1009,0.5254237288135594,0.40487288135593225,-4.138026787575419,0.025619300883706696,Original +1010,0.423728813559322,0.3541431261770245,-2.568187024807373,0.1240305945387,Original +1011,0.3728813559322034,0.29103107344632767,-3.7674177162295606,0.03272349692653532,Original +1012,0.3220338983050847,0.26151129943502827,-3.8711758641214336,0.03050322842853845,Original +1013,0.3389830508474576,0.2489406779661017,-6.249324287797364,0.008266464021748235,Original +1014,0.423728813559322,0.3287429378531073,-1.9757525677542294,0.1426456503116112,Original +1015,0.4915254237288136,0.46002824858757063,-3.229172176597325,0.048246204514222006,Original +1016,0.4067796610169492,0.27019774011299436,-5.5858638786774515,0.011330025903708709,Original +1017,0.3220338983050847,0.26596045197740115,-3.6973851037027248,0.03434015761009596,Original +1018,0.3728813559322034,0.274364406779661,-4.1604734938620584,0.02525593036334952,Original +1019,0.36666666666666664,0.3347457627118644,-1.9898190482959088,0.14070862393531006,Original +1020,0.4067796610169492,0.35437853107344636,-2.9292618143712588,0.06103707112726652,Original +1021,0.3898305084745763,0.3251412429378531,-2.3178660050430633,0.10328535861126524,Original +1022,0.423728813559322,0.3207627118644068,-9.7408025920138,0.002298526341652219,Original +1023,0.3898305084745763,0.2786723163841808,-7.110897442499789,0.005722826489597031,Original +1024,0.4406779661016949,0.32514124293785307,-3.3894381008331114,0.04279257405370464,Original +1025,0.3898305084745763,0.31631355932203387,-4.476394699131555,0.020781977238118643,Original +1026,0.3050847457627119,0.27838983050847455,-1.909090909090913,0.15226214753320294,Original +1027,0.4576271186440678,0.32090395480225986,-4.966780972281921,0.015676121865843336,Original +1028,0.4406779661016949,0.36278248587570616,-8.000108500500483,0.004076417682476059,Original +1029,0.45,0.3771186440677966,-3.8784935044945983,0.030354042786245806,Original +1030,0.3728813559322034,0.3313559322033898,-4.454545454545456,0.04688018499081239,Original +1031,0.423728813559322,0.3502824858757062,-3.8334908600273225,0.03128660551954041,Original +1032,0.5423728813559322,0.3545197740112994,-6.51303802142941,0.0073525543697032375,Original +1033,0.3559322033898305,0.2870762711864407,-4.234130868902486,0.024109305527904665,Original +1034,0.3559322033898305,0.29950564971751414,-1.737938290656899,0.18061180344428132,Original +1035,0.38333333333333336,0.288135593220339,-2.647722058442962,0.07714404224936885,Original +1036,0.4915254237288136,0.35466101694915253,-7.02229226282608,0.005932038385975604,Original +1037,0.423728813559322,0.3632062146892655,-2.200120786201923,0.1151589518445011,Original +1038,0.4406779661016949,0.35444915254237286,-4.143365599990044,0.025532267312653276,Original +1039,0.423728813559322,0.3882062146892655,-2.937104220060188,0.060651717076612384,Original +1040,0.4406779661016949,0.3415960451977401,-3.9152539744439387,0.029618618166276877,Original +1041,0.4745762711864407,0.3203389830508475,-4.2850226450669,0.023356119365012706,Original +1042,0.4067796610169492,0.34173728813559323,-5.289174189736959,0.013184494475533716,Original +1043,0.3898305084745763,0.29103107344632767,-3.585517945697556,0.03714082383617849,Original +1044,0.559322033898305,0.48128531073446335,-3.3967856291561485,0.04256178912971218,Original +1045,0.423728813559322,0.3377824858757062,-3.3339462897065575,0.04458791920005854,Original +1046,0.4666666666666667,0.34745762711864403,-3.119513763428299,0.052491375678345144,Original +1047,0.4915254237288136,0.3121468926553672,-9.141660034508423,0.00276691986404339,Original +1048,0.38333333333333336,0.3305084745762712,-10.796450033846,0.0016997103571064131,Original +1049,0.4576271186440678,0.3672316384180791,-2.274140974459845,0.10751386463711397,Original +1050,0.4745762711864407,0.4007768361581921,-6.242283371885267,0.008292858606378618,Original +1051,0.45,0.37711864406779666,-8.982407047314,0.0029123702976661856,Original +1052,0.43333333333333335,0.288135593220339,-5.089358370900108,0.014661823332742536,Original +1053,0.423728813559322,0.3121468926553672,-3.4347826086956523,0.04139322928919692,Original +1054,0.4067796610169492,0.2641242937853107,-11.821155866809187,0.007080250048172652,Original +1055,0.4915254237288136,0.40084745762711865,-8.576023546581505,0.003332372210409887,Original +1056,0.4067796610169492,0.3121468926553672,-4.822765530016256,0.016987841571810883,Original +1057,0.423728813559322,0.3081214689265537,-10.058098809188749,0.002092572636159546,Original +1058,0.3898305084745763,0.32561205273069677,-3.1284403669724816,0.08877909183169339,Original +1059,0.4915254237288136,0.2994350282485876,-5.277823574803065,0.013263065803865479,Original +1060,0.4067796610169492,0.2915960451977401,-2.792542124181373,0.06827442570311215,Original +1061,0.4067796610169492,0.29237288135593215,-3.5351298218230793,0.07153794820044,Original +1062,0.4576271186440678,0.379590395480226,-2.6941204566469295,0.07415454501482839,Original +1063,0.423728813559322,0.3334039548022599,-3.67018433388411,0.03499558816381141,Original +1064,0.3898305084745763,0.29971751412429376,-3.1708738954340325,0.050446746960355386,Original +1065,0.5423728813559322,0.35014124293785315,-8.799712020856967,0.003091964525888444,Original +1066,0.4666666666666667,0.3644067796610169,-3.63823689123835,0.03578597061066121,Original +1067,0.45,0.3940677966101695,-2.6400000000000006,0.07765595434454427,Original +1068,0.423728813559322,0.346045197740113,-4.6989671174421845,0.01823173060555818,Original +1069,0.4067796610169492,0.3163841807909604,-9.237604307034024,0.002683909079797024,Original +1070,0.4406779661016949,0.3204802259887005,-6.160684868447694,0.008606983305788924,Original +1071,0.45,0.2838983050847458,-7.452597186167861,0.005001392247010432,Original +1072,0.3898305084745763,0.2701271186440678,-5.680855091846446,0.01080903321709519,Original +1073,0.4067796610169492,0.3165960451977401,-4.368550947552518,0.022184572544257958,Original +1074,0.4406779661016949,0.32055084745762713,-6.009400282160132,0.009231944143766397,Original +1075,0.3728813559322034,0.2641242937853107,-3.6019080768824088,0.06917625132966021,Original +1076,0.4067796610169492,0.3248587570621469,-6.653056282246297,0.006920964466083356,Original +1077,0.4067796610169492,0.34173728813559323,-2.5913248074535873,0.08098113762773655,Original +1078,0.4067796610169492,0.35451977401129947,-2.4666666666666663,0.09033115673485469,Original +1079,0.4067796610169492,0.3712570621468927,-2.1556241033270016,0.12006957075894573,Original +1080,0.4067796610169492,0.3207627118644068,-3.6925370142833875,0.03445581989413703,Original +1081,0.423728813559322,0.3080508474576271,-6.565459858828986,0.007186932185349032,Original +1082,0.423728813559322,0.34625706214689267,-2.534027979956117,0.08512323508130125,Original +1083,0.4915254237288136,0.33340395480225987,-6.185153671923991,0.008511170916897891,Original +1084,0.4745762711864407,0.35861581920903957,-6.26717557251908,0.008200036888372677,Original +1085,0.4576271186440678,0.291454802259887,-4.773212266552967,0.01747207617796873,Original +1086,0.4745762711864407,0.3627824858757063,-4.484419263456089,0.020682260794596567,Original +1087,0.5423728813559322,0.37097457627118646,-5.427900519255732,0.012271547258470204,Original +1088,0.3220338983050847,0.29764595103578156,-1.787227246173595,0.21581013089073905,Original +1089,0.4406779661016949,0.32895480225988705,-7.500747797147324,0.0049095659068835425,Original +1090,0.4745762711864407,0.34173728813559323,-9.414301217197753,0.0025394604898687696,Original +1091,0.2711864406779661,0.22768361581920904,-2.8518518518518525,0.06501008999538796,Original +1092,0.38333333333333336,0.2966101694915254,-4.066314336189686,0.026826320116124475,Original +1093,0.4576271186440678,0.32069209039548024,-5.480791700836992,0.011945272816605556,Original +1094,0.3728813559322034,0.3080508474576271,-3.679543437365697,0.034768279257972504,Original +1095,0.3898305084745763,0.29096045197740117,-3.1558437213360135,0.05103452997518892,Original +1096,0.423728813559322,0.3584745762711864,-3.746821857674737,0.0331886297757254,Original +1097,0.4745762711864407,0.3586158192090395,-15.62463762709096,0.0005697357387431864,Original +1098,0.3898305084745763,0.278954802259887,-2.9455339690026325,0.06024089377159606,Original +1099,0.4067796610169492,0.32492937853107345,-3.107327943782055,0.05299174949102144,Original +1100,0.423728813559322,0.35444915254237286,-4.977643693767403,0.01558261256489159,Original +1101,0.3728813559322034,0.314406779661017,-1.5149901793251697,0.2689983409780041,Original +1102,0.4576271186440678,0.417725988700565,-3.7729601826121053,0.03259975536350789,Original +1103,0.3050847457627119,0.253319209039548,-2.5734510578167744,0.0822460968971868,Original +1104,0.3220338983050847,0.2942090395480226,-2.558441558441555,0.2372076297039761,Original +1105,0.4915254237288136,0.27394067796610166,-7.9213555129905915,0.004194695908330953,Original +1106,0.3559322033898305,0.325,-3.3471216807729713,0.04415312534526643,Original +1107,0.4745762711864407,0.40918079096045196,-3.9630593303183272,0.028696120256434428,Original +1108,0.4576271186440678,0.2911016949152542,-3.4316055779260313,0.0414893603948339,Original +1109,0.559322033898305,0.400635593220339,-8.675158396579372,0.0032228830687179013,Original +1110,0.423728813559322,0.3427495291902072,-3.250494467879352,0.08302849336823244,Original +1111,0.4745762711864407,0.38008474576271184,-3.4264466597407415,0.04164606550795528,Original +1112,0.4915254237288136,0.320409604519774,-6.489451756285192,0.007428710958179485,Original +1113,0.4067796610169492,0.36278248587570616,-4.518646959031556,0.02026382990059579,Original +1114,0.559322033898305,0.38806497175141247,-9.914490233494313,0.002182622087591197,Original +1115,0.38333333333333336,0.3135593220338983,-6.377512576754887,0.007804638131363213,Original +1116,0.3559322033898305,0.2616525423728814,-8.372415728614014,0.0035731576383694914,Original +1117,0.423728813559322,0.3500706214689266,-2.6979742584262723,0.07391274673643934,Original +1118,0.45,0.3686440677966102,-7.629306376694646,0.004675053414430021,Original +1119,0.45,0.3728813559322034,-2.1448905695991947,0.12129141727935283,Original +1120,0.4406779661016949,0.3162429378531073,-3.2177938896146965,0.04866601725501004,Original +1121,0.4406779661016949,0.2699152542372881,-6.495776752201798,0.007408187085051301,Original +1122,0.45,0.3432203389830508,-3.505866051946874,0.039314608332933376,Original +1123,0.423728813559322,0.3168785310734463,-3.110976497704088,0.052841304640431436,Original +1124,0.4576271186440678,0.3041666666666667,-4.0423517251725425,0.027245933023793766,Original +1125,0.3898305084745763,0.32627118644067793,-1.6531163063339527,0.24012067051362174,Original +1126,0.3898305084745763,0.325894538606403,-9.506969760375863,0.01088379244125519,Original +1127,0.4406779661016949,0.3924435028248588,-4.386052107630403,0.021948799914748462,Original +1128,0.3050847457627119,0.2489406779661017,-3.2234880893826983,0.04845534939455295,Original +1129,0.36666666666666664,0.2966101694915254,-6.403332465729597,0.00771574670699146,Original +1130,0.4067796610169492,0.3416666666666667,-4.837646025991297,0.016845843964025876,Original +1131,0.4576271186440678,0.3288841807909605,-6.129713906579408,0.0087302972100172,Original +1132,0.4915254237288136,0.3926553672316384,-3.4156502553198673,0.04197645602545185,Original +1133,0.45,0.3771186440677966,-3.6395833778303657,0.03575220047050311,Original +1134,0.3898305084745763,0.24879943502824858,-5.13286206856859,0.014322469145621968,Original +1135,0.3559322033898305,0.25838041431261766,-2.478383793597629,0.13145422697640002,Original +1136,0.4067796610169492,0.3209745762711865,-3.005502067896227,0.05741679214585345,Original +1137,0.3728813559322034,0.29103107344632767,-3.1775375076563694,0.05018888898927705,Original +1138,0.4406779661016949,0.38411016949152543,-2.4611692476881775,0.09077318658872167,Original +1139,0.4067796610169492,0.3458333333333333,-4.644284279214491,0.018818859918736594,Original +1140,0.423728813559322,0.32055084745762713,-4.426845916974451,0.021411691624411183,Original +1141,0.4576271186440678,0.37153954802259886,-3.1115472159109827,0.052817820285826836,Original +1142,0.4576271186440678,0.4094632768361582,-3.037748906538044,0.055966806959142974,Original +1143,0.4576271186440678,0.40494350282485875,-3.713511479606386,0.03395898358756934,Original +1144,0.5,0.36864406779661013,-5.628616053819864,0.011091567639778466,Original +1145,0.4406779661016949,0.3547316384180791,-2.9144455089837993,0.061773504814927487,Original +1146,0.4406779661016949,0.3204802259887006,-7.80733944954129,0.0043741006479608106,Original +1147,0.5,0.3813559322033898,-3.1047293380092054,0.05309922984140076,Original +1148,0.4067796610169492,0.35444915254237286,-2.8491361177963865,0.06515525258058204,Original +1149,0.4067796610169492,0.26560734463276836,-6.047006896155811,0.009071170018542232,Original +1150,0.423728813559322,0.35000000000000003,-2.2152876697379726,0.11354046971526367,Original +1151,0.45,0.3347457627118644,-4.554467869037332,0.019837599235630846,Original +1152,0.3728813559322034,0.34180790960451973,-6.35085296108589,0.007897838244939661,Original +1153,0.4,0.33050847457627114,-2.733333333333336,0.07173922515901705,Original +1154,0.423728813559322,0.346045197740113,-13.339459376998303,0.0009106232895004249,Original +1155,0.3898305084745763,0.2702683615819209,-5.290392517155741,0.013176097120409095,Original +1156,0.4745762711864407,0.3712570621468927,-4.1945951901363205,0.024716200055892694,Original +1157,0.4745762711864407,0.3499293785310735,-3.9622923434347794,0.02871062689188144,Original +1158,0.36666666666666664,0.3305084745762712,-3.3049457887636606,0.045564249549415244,Original +1159,0.4,0.326271186440678,-5.094198380791346,0.014623553591468103,Original +1160,0.4406779661016949,0.3377824858757062,-5.297341314999525,0.013128335633175506,Original +1161,0.4576271186440678,0.3799435028248587,-4.13405315401556,0.02568432777343715,Original +1162,0.423728813559322,0.34625706214689267,-3.451648730864812,0.040887609276162,Original +1163,0.3559322033898305,0.25727401129943506,-3.278863962441917,0.0464655758073765,Original +1164,0.43333333333333335,0.34745762711864403,-4.256851455445149,0.02376922627056341,Original +1165,0.3728813559322034,0.3163841807909604,-3.333333333333336,0.044608278994442765,Original +1166,0.3559322033898305,0.30783898305084745,-3.2916992909941314,0.04601922125686048,Original +1167,0.423728813559322,0.3500706214689266,-3.1298749223282756,0.052070602215977944,Original +1168,0.2711864406779661,0.21920903954802262,-2.9631204814438217,0.09751860335760698,Original +1169,0.423728813559322,0.3543785310734463,-6.117934775079367,0.008777805575829042,Original +1170,0.3728813559322034,0.35021186440677965,-5.6736454811631605,0.010847461425876656,Original +1171,0.423728813559322,0.3694915254237288,-1.777777777777776,0.32619726158657,Original +1172,0.4406779661016949,0.3669491525423729,-3.4506787823167455,0.040916473140808284,Original +1173,0.4,0.2923728813559322,-6.160404875922859,0.00860808784608318,Original +1174,0.5084745762711864,0.32090395480225986,-7.2901037265115765,0.0053285197621824095,Original +1175,0.3898305084745763,0.286864406779661,-3.6376865077145064,0.03579978607225466,Original +1176,0.4406779661016949,0.39237288135593223,-3.433070549496972,0.04144499780826165,Original +1177,0.4067796610169492,0.3372881355932204,-3.3003043252568145,0.08083454343693727,Original +1178,0.3389830508474576,0.2921845574387947,-3.052528420785673,0.09264712636043493,Original +1179,0.4406779661016949,0.37966101694915255,-4.431293675255974,0.02135416398761436,Original +1180,0.4067796610169492,0.3416666666666667,-3.3728798057819063,0.043318525364938,Original +1181,0.3559322033898305,0.295409604519774,-5.351131248404091,0.012766161878428656,Original +1182,0.4406779661016949,0.3373587570621469,-4.004253706903789,0.02793063892749739,Original +1183,0.4067796610169492,0.3082627118644068,-3.4319134544954104,0.04148003219890615,Original +1184,0.423728813559322,0.34187853107344635,-4.233170395077674,0.024123819842227313,Original +1185,0.4915254237288136,0.3927966101694915,-2.484400110285176,0.08892311315035355,Original +1186,0.4745762711864407,0.3501412429378531,-6.005437958250537,0.009249100739188856,Original +1187,0.5254237288135594,0.3246468926553672,-8.378851501317461,0.003565200635279848,Original +1188,0.4915254237288136,0.3753531073446328,-3.92529890966906,0.029421652663864435,Original +1189,0.4576271186440678,0.3626412429378531,-4.119009202618319,0.02593245296220258,Original +1190,0.43333333333333335,0.326271186440678,-2.5654413105709444,0.08282085362511153,Original +1191,0.2711864406779661,0.2235169491525424,-2.8217002208427564,0.06664468710770881,Original +1192,0.559322033898305,0.4976694915254237,-2.725345181640537,0.0722232481118407,Original +1193,0.4067796610169492,0.3484934086629002,-3.5009766627567247,0.07279105093612302,Original +1194,0.3389830508474576,0.3038135593220339,-4.999032554530682,0.015400604884304166,Original +1195,0.3333333333333333,0.30508474576271183,-4.082482904638637,0.02654788546719936,Original +1196,0.4406779661016949,0.3290960451977401,-4.253217026259508,0.023823207735453863,Original +1197,0.3728813559322034,0.3121468926553672,-3.9090909090909123,0.02974030553139564,Original +1198,0.3728813559322034,0.3165960451977401,-3.096482698963895,0.05344214083896168,Original +1199,0.423728813559322,0.3545197740112994,-6.4902085496187105,0.007426251317674435,Original +1200,0.4745762711864407,0.3628531073446328,-2.838043708162847,0.06575238490643358,Original +1201,0.4576271186440678,0.3163841807909605,-4.003203845127177,0.027949819151750765,Original +1202,0.3898305084745763,0.33354519774011293,-2.8433656242933654,0.06546504459353619,Original +1203,0.423728813559322,0.37966101694915255,-4.5490523794544675,0.01990128684164212,Original +1204,0.3728813559322034,0.30783898305084745,-2.3170196256606355,0.10336527760682507,Original +1205,0.4915254237288136,0.3293079096045198,-7.7130494574412145,0.004530195336094142,Original +1206,0.3050847457627119,0.22344632768361583,-3.080575323621966,0.05411151529897513,Original +1207,0.3898305084745763,0.30357815442561203,-3.206327062041345,0.08504610921846024,Original +1208,0.3728813559322034,0.32902542372881355,-4.458405259144846,0.02100779526554658,Original +1209,0.2542372881355932,0.14752824858757063,-6.30893579014757,0.008047357424001828,Original +1210,0.423728813559322,0.35021186440677965,-2.551953540367511,0.08379991236656559,Original +1211,0.3898305084745763,0.3418079096045198,-4.389381125701737,0.02190431958759874,Original +1212,0.288135593220339,0.21532485875706214,-4.8493624334243925,0.016735124561985816,Original +1213,0.3898305084745763,0.2995056497175141,-5.3217343632850165,0.012962453260928666,Original +1214,0.45,0.3432203389830508,-2.896998294604734,0.06265501679556046,Original +1215,0.2711864406779661,0.21532485875706214,-3.1169524568281517,0.05259605209854495,Original +1216,0.4067796610169492,0.3333333333333333,-3.022438607339303,0.05664943058417436,Original +1217,0.4406779661016949,0.26984463276836157,-4.292077854283091,0.023254116236581038,Original +1218,0.4067796610169492,0.30388418079096047,-3.079787687349886,0.054144932145154395,Original +1219,0.3728813559322034,0.32561205273069677,-1.3195730536286363,0.31778111693939937,Original +1220,0.3220338983050847,0.2699152542372881,-2.3318495798744,0.10197574453300426,Original +1221,0.4666666666666667,0.3389830508474576,-3.119096328227442,0.05250841765245064,Original +1222,0.3220338983050847,0.1352401129943503,-8.430881102204697,0.003501725063291807,Original +1223,0.3050847457627119,0.2489406779661017,-7.020021492444736,0.005937531487381006,Original +1224,0.4576271186440678,0.328954802259887,-3.1151717210383336,0.05266898293338368,Original +1225,0.4576271186440678,0.3122881355932204,-3.467950511615431,0.04040635351163493,Original +1226,0.4067796610169492,0.3260828625235405,-3.6801457918777314,0.0665501469345416,Original +1227,0.423728813559322,0.354590395480226,-5.215051241964461,0.01370881147200009,Original +1228,0.3728813559322034,0.3038135593220339,-9.817377185403636,0.002246445968285984,Original +1229,0.3220338983050847,0.26596045197740115,-2.465458081419365,0.0904281089259112,Original +1230,0.5084745762711864,0.36264124293785316,-5.251461666642595,0.0134479218636803,Original +1231,0.3728813559322034,0.27443502824858756,-3.458769578971492,0.04067649752950825,Original +1232,0.4406779661016949,0.34173728813559323,-3.108570507260323,0.052940452969860205,Original +1233,0.45,0.3220338983050847,-3.55910413197229,0.037844293787820955,Original +1234,0.3559322033898305,0.31652542372881354,-2.9875904328450478,0.058242586475556184,Original +1235,0.4745762711864407,0.4096045197740113,-2.5245779797628782,0.0858312019877726,Original +1236,0.3389830508474576,0.3089453860640301,-2.8553936637391883,0.10388662348248497,Original +1237,0.3898305084745763,0.3458333333333333,-2.306948594944211,0.10432200362127822,Original +1238,0.3898305084745763,0.3038135593220339,-7.137099922436499,0.005662825302674034,Original +1239,0.3728813559322034,0.18559322033898304,-10.368238367141146,0.0019142559726229866,Original +1240,0.3728813559322034,0.2786016949152542,-6.156258728408069,0.008624465754537623,Original +1241,0.3728813559322034,0.2956920903954802,-2.834146743848625,0.06596378750306289,Original +1242,0.4067796610169492,0.34187853107344635,-3.891797498256559,0.030085204803105746,Original +1243,0.38333333333333336,0.3389830508474576,-2.266099806569284,0.1083141230940181,Original +1244,0.3220338983050847,0.25741525423728817,-2.4557675307968445,0.09121010163141306,Original +1245,0.5254237288135594,0.39258474576271185,-3.5522131403755797,0.03803061001063327,Original +1246,0.4406779661016949,0.31207627118644066,-5.440282569102571,0.012194130448693096,Original +1247,0.36666666666666664,0.32627118644067793,-3.788162541206027,0.032263411126476765,Original +1248,0.38333333333333336,0.34745762711864403,-3.2791258997889505,0.046456412128927795,Original +1249,0.3220338983050847,0.25741525423728817,-5.952990444986046,0.009480187104661204,Original +1250,0.3389830508474576,0.3258003766478343,-3.4999999999999902,0.07282735005446969,Original +1251,0.3559322033898305,0.2783898305084746,-3.9414887073599294,0.029107725320715257,Original +1252,0.4,0.364406779661017,-2.193378465041791,0.1158873271135981,Original +1253,0.3389830508474576,0.28255649717514125,-3.1504339973019846,0.05124820108543329,Original +1254,0.4745762711864407,0.41377118644067795,-2.677569682825962,0.07520419613663175,Original +1255,0.3559322033898305,0.24491525423728813,-6.057634803744621,0.00902640078253155,Original +1256,0.4576271186440678,0.29117231638418084,-4.3575806009385145,0.022334035979091265,Original +1257,0.3898305084745763,0.32916666666666666,-10.509267049042842,0.001839820961868861,Original +1258,0.38333333333333336,0.2966101694915254,-4.298859857965993,0.023156605272933835,Original +1259,0.423728813559322,0.2870056497175142,-17.37458743854924,0.0004155011309011471,Original +1260,0.3050847457627119,0.2303201506591337,-5.066892497031914,0.0368133974917626,Original +1261,0.4576271186440678,0.2956920903954802,-5.945744266829731,0.009512707270210679,Original +1262,0.3220338983050847,0.2750470809792844,-2.0432809561528478,0.1777389669818575,Original +1263,0.4576271186440678,0.3129943502824859,-2.399133282225983,0.09594924200933898,Original +1264,0.4166666666666667,0.3050847457627119,-4.8621138936836426,0.016615697449670994,Original +1265,0.3389830508474576,0.2699858757062147,-3.0417250416423958,0.055791220683953764,Original +1266,0.3898305084745763,0.30348399246704333,-4.849951400603131,0.039981081861906576,Original +1267,0.4,0.3093220338983051,-4.068509688367149,0.02678829448291478,Original +1268,0.423728813559322,0.2828389830508474,-12.090909090909088,0.0012175905817395098,Original +1269,0.4067796610169492,0.28276836158192087,-4.398585621119437,0.021781940473571012,Original +1270,0.3559322033898305,0.2784604519774011,-3.6540488693517794,0.03539195989892865,Original +1271,0.35,0.2754237288135593,-3.195601372491276,0.049498206131116385,Original +1272,0.3898305084745763,0.30357815442561203,-3.206327062041345,0.08504610921846024,Original +1273,0.4,0.3347457627118644,-2.6807959017177474,0.07499815239256642,Original +1274,0.4406779661016949,0.35430790960451974,-2.9187879140660207,0.06155652231610408,Original +1275,0.3220338983050847,0.24901129943502825,-2.864760573339222,0.06432558093621543,Original +1276,0.5423728813559322,0.3375,-8.30959245705665,0.0036520779681317537,Original +1277,0.288135593220339,0.22351694915254236,-3.3098123366183496,0.04539853318110197,Original +1278,0.4067796610169492,0.3161723163841808,-2.483483730018013,0.0889952126349582,Original +1279,0.31666666666666665,0.2754237288135593,-3.8676344826299247,0.030575765124777585,Original +1280,0.35,0.2966101694915254,-3.6373066958946425,0.03580932390070352,Original +1281,0.3,0.2584745762711864,-2.869146214468688,0.06409507369957781,Original +1282,0.4067796610169492,0.3423728813559322,-2.203747318127119,0.15839077808830893,Original +1283,0.288135593220339,0.2278954802259887,-2.1351104338488116,0.12241769751046185,Original +1284,0.3389830508474576,0.28269774011299437,-3.521966119583988,0.03886239529216442,Original +1285,0.23728813559322035,0.19077212806026367,-1.9859386750277492,0.1854304501080007,Original +1286,0.2542372881355932,0.21942090395480224,-37.92307692307706,4.033425013157966e-05,Original +1287,0.3728813559322034,0.2909604519774011,-2.6148419405355545,0.07935310501190229,Original +1288,0.4166666666666667,0.31779661016949157,-6.831300510639733,0.0064184892478205335,Original +1289,0.3898305084745763,0.23241525423728815,-4.29193057069012,0.02325623974857282,Original +1290,0.3898305084745763,0.3377118644067797,-3.1733791192340908,0.0503496079687739,Original +1291,0.3220338983050847,0.2364406779661017,-3.377315745886016,0.04317682241293437,Original +1292,0.5254237288135594,0.3247175141242937,-5.643467443294131,0.011010270079289456,Original +1293,0.4406779661016949,0.34611581920903955,-7.873610453398626,0.004268617658520372,Original +1294,0.423728813559322,0.3162429378531073,-2.825094808781018,0.06645811724840166,Original +1295,0.3389830508474576,0.26617231638418076,-2.337868480725624,0.10141825639016255,Original +1296,0.2542372881355932,0.18975988700564972,-3.2917415848871014,0.04601775946765137,Original +1297,0.3389830508474576,0.25741525423728817,-3.8324745624304253,0.03130808733647537,Original +1298,0.576271186440678,0.32902542372881355,-9.542627417316496,0.0024409490317057526,Original +1299,0.3389830508474576,0.27860169491525427,-2.340709452706567,0.10115640234950059,Original +1300,0.3220338983050847,0.27005649717514124,-7.413571269033491,0.005077486364715219,Original +1301,0.3559322033898305,0.30798022598870056,-3.889154254622231,0.030138373467386664,Original +1302,0.3728813559322034,0.30254237288135594,-27.6666666666665,0.023000340589345674,Original +1303,0.3728813559322034,0.27464689265536724,-3.0029751452311446,0.05753239842963967,Original +1304,0.2711864406779661,0.24463276836158193,-2.631494801492514,0.07822464952325936,Original +1305,0.43333333333333335,0.3432203389830508,-2.8168388126916613,0.06691301311594693,Original +1306,0.45,0.31779661016949157,-5.039254603264428,0.015065771599115848,Original +1307,0.3389830508474576,0.2701271186440678,-3.9729784680823,0.028509358260687255,Original +1308,0.4576271186440678,0.2951977401129944,-6.340155233829669,0.00793564788180792,Original +1309,0.3728813559322034,0.2615819209039548,-3.256754496379134,0.04724737989320142,Original +1310,0.3728813559322034,0.2699152542372881,-5.1240286090443545,0.014390539846078056,Original +1311,0.4406779661016949,0.37563559322033896,-2.224345699469886,0.11258691448891019,Original +1312,0.3728813559322034,0.29783427495291903,-3.5957489244818297,0.06938932086375887,Original +1313,0.23728813559322035,0.18568738229755177,-1.9376663567761192,0.19225685599558717,Original +1314,0.3333333333333333,0.2796610169491526,-3.6565517048676255,0.035330102302749224,Original +1315,0.26666666666666666,0.1906779661016949,-2.7138510050026126,0.07292684266802282,Original +1316,0.4745762711864407,0.37570621468926557,-2.8482596056990577,0.06520219060265608,Original +1317,0.3728813559322034,0.29103107344632767,-4.887164066279016,0.016384294164034276,Original +1318,0.23333333333333334,0.1483050847457627,-2.866666666666667,0.06422527059785652,Original +1319,0.3728813559322034,0.29209039548022603,-7.654614504282301,0.016642001057945596,Original +1320,0.423728813559322,0.2913135593220339,-4.79001040774592,0.017305937126765883,Original +1321,0.4406779661016949,0.28700564971751413,-6.922217740406311,0.006180622722235997,Original +1322,0.4745762711864407,0.3292372881355933,-6.337740226443206,0.007944216283270866,Original +1323,0.2711864406779661,0.2192090395480226,-2.472975320624164,0.08982713951273004,Original +1324,0.4406779661016949,0.33319209039548026,-6.046351066103405,0.00907394218732086,Original +1325,0.3728813559322034,0.269774011299435,-2.6462479326196116,0.0772414424024133,Original +1326,0.423728813559322,0.30798022598870056,-6.24495382880618,0.008282834737265843,Original +1327,0.3898305084745763,0.29936440677966103,-4.542638876598001,0.019977054761155088,Original +1328,0.4666666666666667,0.3305084745762712,-3.092031441659996,0.05362839188122225,Original +1329,0.3389830508474576,0.2662429378531074,-2.0881715296001104,0.12800003155007977,Original +1330,0.423728813559322,0.29936440677966103,-2.5346106753467175,0.08507981646292648,Original +1331,0.3389830508474576,0.2617231638418079,-4.608368949689674,0.019217889743118922,Original +1332,0.4576271186440678,0.3415960451977401,-3.626931567328918,0.036071124625796494,Original +1333,0.4915254237288136,0.38022598870056495,-2.535033549783168,0.08504832368725805,Original +1334,0.423728813559322,0.3209039548022599,-3.4248301792021634,0.04169532187000911,Original +1335,0.4067796610169492,0.3545197740112994,-4.1111111111111125,0.02606395678518873,Original +1336,0.5254237288135594,0.32457627118644067,-6.742138664692472,0.006663608853032706,Original +1337,0.36666666666666664,0.288135593220339,-2.6750562472452653,0.07536520355046247,Original +1338,0.4406779661016949,0.3627118644067797,-3.6442193578369366,0.03563624031042606,Original +1339,0.4067796610169492,0.28241525423728814,-3.0409065774469273,0.0558273074325418,Original +1340,0.3050847457627119,0.2319915254237288,-6.013276450562215,0.009215200973280897,Original +1341,0.4067796610169492,0.34576271186440677,-2.4308053376742573,0.09326286520252099,Original +1342,0.4745762711864407,0.40091807909604515,-2.985011716688378,0.05836269929186945,Original +1343,0.288135593220339,0.2192090395480226,-4.357142857142859,0.02234002693058057,Original +1344,0.288135593220339,0.24456214689265537,-2.609888745022117,0.07969262184669647,Original +1345,0.5254237288135594,0.47238700564971753,-1.8841000222390059,0.15606129783472963,Original +1346,0.4406779661016949,0.3461864406779661,-5.059912204313382,0.014897489104880094,Original +1347,0.4666666666666667,0.3093220338983051,-3.3757575757575755,0.04322652987968803,Original +1348,0.4576271186440678,0.3797316384180791,-2.9703710359097775,0.05905056214606797,Original +1349,0.3559322033898305,0.2362523540489642,-3.955281402843921,0.05837978400303097,Original +1350,0.3389830508474576,0.29764595103578156,-4.346534653465347,0.04906822409381993,Original +1351,0.3559322033898305,0.2911723163841808,-4.423761578539075,0.021451702572010452,Original +1352,0.3728813559322034,0.27838983050847455,-3.7753252988406856,0.03254713415412944,Original +1353,0.3898305084745763,0.3586158192090395,-2.541573388991179,0.08456310051618006,Original +1354,0.3220338983050847,0.2318502824858757,-2.7486274334487737,0.07082370041149343,Original +1355,0.3559322033898305,0.29936440677966103,-2.1631195885543937,0.11922505386124203,Original +1356,0.45,0.30508474576271183,-12.091525958289973,0.0012174072403681394,Original +1357,0.3728813559322034,0.29103107344632767,-3.7674177162295606,0.03272349692653532,Original +1358,0.423728813559322,0.35861581920903957,-3.110819906677478,0.05284775046942044,Original +1359,0.3220338983050847,0.2809792843691149,-5.736842105263147,0.02906640150529952,Original +1360,0.3050847457627119,0.2526365348399247,-3.4171779141104293,0.07600312612521545,Original +1361,0.4166666666666667,0.3220338983050848,-3.53121005385469,0.03860574928821704,Original +1362,0.3559322033898305,0.314406779661017,-1.800059978008079,0.2136563487360759,Original +1363,0.4406779661016949,0.3840395480225989,-2.2968348396222584,0.10529360084626542,Original +1364,0.3050847457627119,0.2153954802259887,-4.628137283451674,0.01899691387142354,Original +1365,0.3389830508474576,0.2701271186440678,-2.5462474269165756,0.08421839435807502,Original +1366,0.36666666666666664,0.288135593220339,-3.421943449225448,0.04178346889141796,Original +1367,0.3559322033898305,0.29992937853107343,-1.9390607807005154,0.1478476458083794,Original +1368,0.3559322033898305,0.26935028248587567,-1.643431635388741,0.34799773853532184,Original +1369,0.3220338983050847,0.22761299435028248,-4.4183200488016485,0.021522528086765566,Original +1370,0.423728813559322,0.2528248587570622,-5.471809142918229,0.011999877727690598,Original +1371,0.4166666666666667,0.3093220338983051,-13.229901852633054,0.0009331248455100845,Original +1372,0.4166666666666667,0.3474576271186441,-4.715027198381945,0.018063824737018258,Original +1373,0.3050847457627119,0.2699858757062147,-5.869851287372005,0.009862263796097724,Original +1374,0.36666666666666664,0.2923728813559322,-5.133234383709284,0.014319609315535732,Original +1375,0.3728813559322034,0.3040960451977401,-2.8752211601887643,0.06377747883646023,Original +1376,0.3728813559322034,0.3248587570621469,-1.9896995023342197,0.14072495503830776,Original +1377,0.3559322033898305,0.2953389830508475,-4.1824309711049805,0.024906888121385386,Original +1378,0.3898305084745763,0.27429378531073445,-4.389736277617274,0.021899581152473327,Original +1379,0.4576271186440678,0.30360169491525424,-6.426920856371511,0.007635697442918866,Original +1380,0.3559322033898305,0.2955508474576271,-3.519233773599044,0.03893867159865646,Original +1381,0.423728813559322,0.3082627118644068,-4.022350177849458,0.02760267623485285,Original +1382,0.3898305084745763,0.34821092278719395,-4.510204081632659,0.045808057902316804,Original +1383,0.423728813559322,0.3258003766478343,-4.913538149119954,0.039012347759055314,Original +1384,0.3050847457627119,0.24887005649717514,-3.297939899442541,0.04580416394678644,Original +1385,0.3898305084745763,0.29209039548022603,-9.26047768699887,0.011460849730983888,Original +1386,0.3898305084745763,0.3250706214689265,-4.167794334654786,0.025138858812902672,Original +1387,0.4576271186440678,0.3121468926553672,-3.4545091251635194,0.04080264000619044,Original +1388,0.45,0.385593220338983,-5.066666666666667,0.014842997288171941,Original +1389,0.3559322033898305,0.2530367231638418,-3.0699274770848146,0.05456547495842803,Original +1390,0.3050847457627119,0.2402542372881356,-2.253658340717321,0.10956651809067314,Original +1391,0.4,0.3220338983050848,-2.732807218615621,0.07177097972724268,Original +1392,0.4067796610169492,0.32069209039548024,-7.1384266143873925,0.005659809403554773,Original +1393,0.3050847457627119,0.24152542372881355,-5.9603956067927,0.027012812355076195,Original +1394,0.3898305084745763,0.29145480225988696,-3.4168337494506282,0.04194007659421182,Original +1395,0.1864406779661017,0.13495762711864406,-2.257918405893237,0.10913572969388878,Original +1396,0.3050847457627119,0.23644067796610171,-3.233110040049279,0.04810198076292131,Original +1397,0.3559322033898305,0.27429378531073445,-2.907350507765195,0.062130092881679595,Original +1398,0.288135593220339,0.24053672316384178,-2.365917408343781,0.0988685692030821,Original +1399,0.3559322033898305,0.24915254237288134,-3.4558397655933613,0.04076318933119888,Original +1400,0.423728813559322,0.34611581920903955,-2.894903284380277,0.06276192114601883,Original +1401,0.3898305084745763,0.34201977401129946,-2.394619302503736,0.09633975086570609,Original +1402,0.3559322033898305,0.27853107344632766,-5.181815900134804,0.013952747029258906,Original +1403,0.43333333333333335,0.3686440677966102,-4.46962233410428,0.020866618588713408,Original +1404,0.423728813559322,0.3754943502824859,-3.4249037287283453,0.041693079107582236,Original +1405,0.4915254237288136,0.3121468926553672,-9.141660034508423,0.00276691986404339,Original +1406,0.4745762711864407,0.3968926553672316,-2.528902694294367,0.08550631275341908,Original +1407,0.4,0.25,-14.066533632030769,0.0007781509499729613,Original +1408,0.423728813559322,0.3584039548022599,-3.1685329501250488,0.050537730061605154,Original +1409,0.4745762711864407,0.38848870056497176,-3.0811823304553716,0.05408577968425568,Original +1410,0.4745762711864407,0.40529661016949153,-3.1510846069279967,0.05122244392981766,Original +1411,0.3728813559322034,0.2995762711864407,-6.934810075795833,0.006148597978661057,Original +1412,0.3728813559322034,0.34180790960451973,-6.35085296108589,0.007897838244939661,Original +1413,0.5084745762711864,0.34209039548022596,-5.7423119859693115,0.010488586403531558,Original +1414,0.5166666666666667,0.3686440677966102,-6.342784542369049,0.007926332946572029,Original +1415,0.4745762711864407,0.3,-4.012310343727843,0.027784012310922687,Original +1416,0.4915254237288136,0.4219632768361582,-9.741023075752745,0.0022983741002138604,Original +1417,0.5254237288135594,0.4518361581920904,-2.3115409557535824,0.10388441475596431,Original +1418,0.3728813559322034,0.31257062146892656,-2.208880071004421,0.11422087778664086,Original +1419,0.3898305084745763,0.3671610169491525,-3.9629629629629703,0.02869794241127541,Original +1420,0.4067796610169492,0.3545197740112994,-4.900769721140663,0.01626036695364968,Original +1421,0.48333333333333334,0.4194915254237288,-2.0960998088359606,0.12703618205803208,Original +1422,0.4745762711864407,0.4010593220338983,-4.205211154248234,0.024551320079546985,Original +1423,0.5084745762711864,0.4175141242937853,-2.3260164970888964,0.10251957254301898,Original +1424,0.4915254237288136,0.3879237288135593,-2.9208912467506574,0.06145176525841497,Original +1425,0.45,0.41101694915254233,-2.231327750334267,0.11185847276008758,Original +1426,0.3728813559322034,0.2995056497175141,-2.210333134870083,0.11406615150921069,Original +1427,0.3559322033898305,0.29774011299435027,-3.952777215734684,0.05844741686027941,Original +1428,0.4,0.3432203389830508,-3.021617032571375,0.05668635618881769,Original +1429,0.4067796610169492,0.3079802259887006,-34.12195121951229,5.5338575696591484e-05,Original +1430,0.4666666666666667,0.4110169491525424,-6.858659644654502,0.006345660358547117,Original +1431,0.4067796610169492,0.33799435028248587,-1.8646963468092161,0.15908733357598287,Original +1432,0.4406779661016949,0.39887005649717516,-2.8461538461538454,0.1044600519927322,Original +1433,0.5254237288135594,0.3629237288135593,-10.808536082221455,0.0016941290870239134,Original +1434,0.423728813559322,0.3291666666666667,-5.281280870229153,0.01323906907450259,Original +1435,0.4067796610169492,0.2950564971751412,-2.5561253487432714,0.08349557275673589,Original +1436,0.4576271186440678,0.37132768361581925,-12.168997094322684,0.0011946701728581847,Original +1437,0.4745762711864407,0.3586158192090395,-3.8295559807933683,0.03136988366637992,Original +1438,0.4576271186440678,0.3669491525423728,-5.570197507360285,0.01141908670501291,Original +1439,0.3559322033898305,0.30819209039548023,-2.607416226140155,0.07986277130041312,Original +1440,0.48333333333333334,0.3813559322033898,-4.548172491877701,0.019911659585532055,Original +1441,0.3559322033898305,0.29124293785310734,-3.602392726796967,0.036700127748095945,Original +1442,0.4915254237288136,0.3882062146892656,-7.414941042518773,0.00507478976632389,Original +1443,0.3728813559322034,0.27888418079096045,-2.9282749062301643,0.06108578193839565,Original +1444,0.4745762711864407,0.3502824858757062,-6.487446070815471,0.007435234784132052,Original +1445,0.4915254237288136,0.4301553672316384,-2.6869021956079524,0.07461008165667392,Original +1446,0.43333333333333335,0.3220338983050848,-2.888948165920659,0.06306704716766874,Original +1447,0.4067796610169492,0.3163841807909605,-13.063945294843638,0.0009686387721898685,Original +1448,0.423728813559322,0.2754237288135593,-11.666666666666663,0.007266951354550622,Original +1449,0.5084745762711864,0.41779661016949154,-2.9499240992842286,0.060028321215766844,Original +1450,0.4,0.364406779661017,-2.193378465041791,0.1158873271135981,Original +1451,0.45,0.364406779661017,-3.2485009805585183,0.04754349406047413,Original +1452,0.4406779661016949,0.3038841807909604,-7.310266791966458,0.0052864231861055435,Original +1453,0.3898305084745763,0.303954802259887,-5.811865258054232,0.010140767780751686,Original +1454,0.423728813559322,0.3931261770244821,-2.400829779247631,0.1383734677102449,Original +1455,0.4067796610169492,0.3542372881355932,-2.708482575649219,0.0732583731798295,Original +1456,0.3898305084745763,0.3334039548022599,-3.7816368562219433,0.0324072409590771,Original +1457,0.4576271186440678,0.3794491525423729,-2.2802263709416946,0.10691298923861156,Original +1458,0.423728813559322,0.35416666666666663,-2.7464616338694436,0.07095246240968993,Original +1459,0.43333333333333335,0.3728813559322034,-2.118915741970083,0.1243102693621582,Original +1460,0.423728813559322,0.37153954802259886,-2.386591892485171,0.09703900448133006,Original +1461,0.4576271186440678,0.3502824858757062,-8.101627221513192,0.003930361543241993,Original +1462,0.3898305084745763,0.33742937853107347,-3.6447947158122775,0.035621882387353185,Original +1463,0.4915254237288136,0.2951977401129944,-9.61483104773844,0.002387739072267894,Original +1464,0.4067796610169492,0.31207627118644066,-4.941615370705469,0.015895579161855126,Original +1465,0.4745762711864407,0.35903954802259885,-3.4967484138936817,0.03957367708757962,Original +1466,0.4666666666666667,0.36864406779661013,-5.216423434339635,0.013698860156045016,Original +1467,0.43333333333333335,0.3813559322033898,-10.623244953089115,0.0017824441583808043,Original +1468,0.3728813559322034,0.2919020715630885,-2.841519981956887,0.10474932255486497,Original +1469,0.4067796610169492,0.33681732580037665,-2.5438493495451917,0.12598335802475616,Original +1470,0.423728813559322,0.3456920903954802,-3.165877220586101,0.0506411985161682,Original +1471,0.423728813559322,0.29774011299435027,-5.144479879486528,0.03576976981268908,Original +1472,0.6,0.4957627118644068,-4.919999999999999,0.016087280685965905,Original +1473,0.4745762711864407,0.36292372881355933,-2.9112467789048893,0.06193395217425993,Original +1474,0.4067796610169492,0.28721751412429375,-4.6372424549345395,0.018896243004971457,Original +1475,0.3898305084745763,0.3077683615819209,-3.416406029443321,0.04195321965391951,Original +1476,0.4576271186440678,0.3416666666666667,-6.968395313620221,0.006064246319147875,Original +1477,0.4067796610169492,0.325,-2.3275547317374796,0.10237581960629237,Original +1478,0.423728813559322,0.38201506591337103,-2.838985414963086,0.10490802421935423,Original +1479,0.4915254237288136,0.3797316384180791,-6.404709068596953,0.007711044857755044,Original +1480,0.3898305084745763,0.2909604519774011,-3.081578172139684,0.054069005258383575,Original +1481,0.55,0.40254237288135597,-5.266283362867523,0.013343577598251742,Original +1482,0.4067796610169492,0.3587570621468927,-3.544745038970269,0.038233853222009985,Original +1483,0.423728813559322,0.3248587570621469,-2.2831482556870473,0.10662592307782301,Original +1484,0.36666666666666664,0.2923728813559322,-4.252457958970371,0.023834502002819766,Original +1485,0.4576271186440678,0.40084745762711865,-3.941176470588236,0.029113738898400077,Original +1486,0.423728813559322,0.32090395480225986,-5.31174648930179,0.013030041853782981,Original +1487,0.3728813559322034,0.3204802259887006,-2.1906690113295393,0.11618158740373133,Original +1488,0.4406779661016949,0.325,-3.9643008779307634,0.028672657850254017,Original +1489,0.4915254237288136,0.3544491525423729,-7.463121733660972,0.00498112898453179,Original +1490,0.5,0.3686440677966102,-4.69122943243946,0.018313350909407743,Original +1491,0.4576271186440678,0.3626412429378531,-2.5444250028895907,0.08435259068095222,Original +1492,0.576271186440678,0.4848870056497175,-2.3523540362420694,0.10009165403065308,Original +1493,0.3559322033898305,0.303954802259887,-3.517707919348613,0.038981350469306286,Original +1494,0.4576271186440678,0.3038135593220339,-4.198379191556001,0.024657266011800877,Original +1495,0.4067796610169492,0.32923728813559316,-5.019825255742889,0.015226316305783647,Original +1496,0.423728813559322,0.33757062146892663,-1.955570219094377,0.18968621213880818,Original +1497,0.5084745762711864,0.4048728813559322,-5.068605007676937,0.014827407662859303,Original +1498,0.3898305084745763,0.3425612052730697,-2.4888108666663245,0.13056092060471672,Original +1499,0.423728813559322,0.37146892655367236,-3.9219921560235247,0.029486306559984257,Original +1500,0.4915254237288136,0.35035310734463276,-3.355609536673555,0.043875869462308204,Original +1501,0.5254237288135594,0.36737288135593216,-5.454903356437624,0.012103536343441582,Original +1502,0.288135593220339,0.2362994350282486,-7.416033307666337,0.005072640837140369,Original +1503,0.3728813559322034,0.3206920903954802,-4.327561335547406,0.022749742422749963,Original +1504,0.3559322033898305,0.29957627118644065,-7.052288308361696,0.005860100906942432,Original +1505,0.4915254237288136,0.3422316384180791,-2.414469415876127,0.09463683116855957,Original +1506,0.4745762711864407,0.354590395480226,-6.260853478619376,0.008223481825561816,Original +1507,0.4745762711864407,0.3882062146892655,-4.699364429896171,0.018227552356258447,Original +1508,0.4406779661016949,0.31624293785310736,-3.384433655570972,0.042950673824966874,Original +1509,0.4406779661016949,0.31235875706214694,-8.330919086612344,0.003625030826573087,Original +1510,0.4406779661016949,0.28700564971751413,-14.665257155667614,0.0006876691247397437,Original +1511,0.3898305084745763,0.3415960451977401,-3.006477663714325,0.05737223684387089,Original +1512,0.3898305084745763,0.31193502824858754,-2.8294238473612916,0.06622113261732844,Original +1513,0.4166666666666667,0.3601694915254237,-3.233808333817773,0.0480764627944047,Original +1514,0.4666666666666667,0.35593220338983056,-4.438525973846813,0.021261047550038326,Original +1515,0.43333333333333335,0.3432203389830508,-2.8168388126916613,0.06691301311594693,Original +1516,0.423728813559322,0.3334745762711865,-3.3752877168731676,0.043241533194103954,Original +1517,0.4576271186440678,0.4176553672316384,-2.608655045302455,0.07977746445984121,Original +1518,0.4406779661016949,0.3330508474576271,-3.8022134355217396,0.031956488835427684,Original +1519,0.3728813559322034,0.26581920903954803,-6.147383047873269,0.008659663805844972,Original +1520,0.5254237288135594,0.3882062146892655,-7.465956735313377,0.004975689184389457,Original +1521,0.43333333333333335,0.3728813559322034,-2.4230728530694363,0.09391016538665153,Original +1522,0.3559322033898305,0.31207627118644066,-2.1523998630107477,0.12043504718327508,Original +1523,0.3898305084745763,0.30856873822975517,-2.102946127279898,0.1701873208222798,Original +1524,0.3728813559322034,0.2913135593220339,-4.4679931695295805,0.020887046587591118,Original +1525,0.48333333333333334,0.3686440677966102,-5.413333333333333,0.012363451013336186,Original +1526,0.45,0.3516949152542373,-3.2276227144907716,0.048303103292659674,Original +1527,0.4406779661016949,0.3209745762711864,-3.5295270186824976,0.038652315352543654,Original +1528,0.3898305084745763,0.3119350282485876,-2.259178910141389,0.10900865631385685,Original +1529,0.4166666666666667,0.326271186440678,-5.6348858004839935,0.011057151074446126,Original +1530,0.4067796610169492,0.2870762711864407,-5.605795079572491,0.011218029506615877,Original +1531,0.5423728813559322,0.3502824858757062,-4.93323850074808,0.015969516750107843,Original +1532,0.423728813559322,0.3292372881355932,-1.9315505529107941,0.14893966449897095,Original +1533,0.5254237288135594,0.4470338983050848,-2.370301890762351,0.0984770870147508,Original +1534,0.4576271186440678,0.33340395480225987,-2.965463727554792,0.0592833948188603,Original +1535,0.4,0.3347457627118644,-3.0800000000000014,0.05413592185295247,Original +1536,0.423728813559322,0.4044256120527307,-2.4634512493170466,0.13274843347385198,Original +1537,0.4,0.326271186440678,-2.8103535287436237,0.0672730683623227,Original +1538,0.5166666666666667,0.41949152542372886,-2.4681862514695623,0.09020943675168575,Original +1539,0.5666666666666667,0.423728813559322,-4.404164695679157,0.021708194051045772,Original +1540,0.5254237288135594,0.40091807909604515,-8.284302459961115,0.0036844984307791845,Original +1541,0.4406779661016949,0.3587570621468926,-3.775478418443893,0.032543731166538116,Original +1542,0.3898305084745763,0.35854519774011306,-1.7390744938729776,0.18040454692234978,Original +1543,0.423728813559322,0.35021186440677965,-3.0253320224395073,0.05651962666233991,Original +1544,0.4576271186440678,0.3923022598870057,-2.2432895898859275,0.11062363137762433,Original +1545,0.4915254237288136,0.3714689265536723,-5.879572942861632,0.009816556345478428,Original +1546,0.4745762711864407,0.3338276836158192,-3.2632296876876485,0.047016699886531045,Original +1547,0.45,0.35169491525423724,-12.115804854516572,0.0012102202443320875,Original +1548,0.4745762711864407,0.3627824858757063,-8.098378653528943,0.003934927255862458,Original +1549,0.423728813559322,0.32062146892655363,-9.054535625012225,0.002845269537083193,Original +1550,0.4666666666666667,0.3686440677966102,-6.772746506262815,0.006578095712086025,Original +1551,0.5084745762711864,0.40925141242937857,-4.746377851626824,0.01774180468795205,Original +1552,0.4666666666666667,0.3305084745762712,-5.565656594987991,0.011445072212549055,Original +1553,0.38333333333333336,0.3220338983050847,-2.557369525291349,0.08340507155260364,Original +1554,0.559322033898305,0.40557909604519776,-3.3491158311330995,0.044087787022349285,Original +1555,0.4745762711864407,0.34576271186440677,-6.516946235415337,0.007340034612930771,Original +1556,0.3898305084745763,0.28255649717514125,-2.8447438754418775,0.06539088569746056,Original +1557,0.3728813559322034,0.3204802259887006,-2.685797986127019,0.0746800724946613,Original +1558,0.3898305084745763,0.3375706214689266,-3.7567808109943908,0.03296266830646462,Original +1559,0.423728813559322,0.3309792843691149,-1.7858245155746162,0.2160472917948211,Original +1560,0.4915254237288136,0.4471045197740113,-3.0677087606277684,0.054660670246442895,Original +1561,0.4576271186440678,0.35918079096045197,-2.275676500715352,0.10736186186928126,Original +1562,0.5084745762711864,0.29936440677966103,-9.423964347365773,0.0025318610514215717,Original +1563,0.423728813559322,0.35847457627118645,-3.2665066947432746,0.04690049922235536,Original +1564,0.5084745762711864,0.3969632768361582,-2.896194824692244,0.06269598939057865,Original +1565,0.35,0.3177966101694915,-7.600000000000018,0.004727189949278468,Original +1566,0.4,0.364406779661017,-1.7643529058752923,0.17586309782892548,Original +1567,0.4067796610169492,0.34152542372881356,-2.7297642578116172,0.07195498560460242,Original +1568,0.3898305084745763,0.3419491525423729,-2.59993484464519,0.08038034833840463,Original +1569,0.4915254237288136,0.38834745762711864,-7.704320406910871,0.00454501569408191,Original +1570,0.3898305084745763,0.32033898305084746,-3.891547782585247,0.06013719541915213,Original +1571,0.4576271186440678,0.3543785310734463,-5.771672200284073,0.010339886184267465,Original +1572,0.4,0.3474576271186441,-4.802499349297195,0.017183746707078684,Original +1573,0.4067796610169492,0.3504237288135593,-3.262064802178429,0.04705809381582309,Original +1574,0.4067796610169492,0.34611581920903955,-2.3420605578207687,0.10103215816449211,Original +1575,0.5084745762711864,0.4430790960451977,-12.700432204420535,0.001052946749526643,Original +1576,0.423728813559322,0.36292372881355933,-5.327313464522444,0.012924898405343976,Original +1577,0.5084745762711864,0.38425141242937855,-4.698424427455757,0.018237439696403804,Original +1578,0.4576271186440678,0.3882768361581921,-6.446590136835602,0.007569780791386365,Original +1579,0.4915254237288136,0.328954802259887,-7.995948131756914,0.004082555066940009,Original +1580,0.4067796610169492,0.3249293785310734,-4.059276987931347,0.026948683537732478,Original +1581,0.4166666666666667,0.3601694915254237,-4.444444444444445,0.0211852376832342,Original +1582,0.5166666666666667,0.3135593220338983,-6.483004673258841,0.007449708002644034,Original +1583,0.4067796610169492,0.3586158192090395,-3.921613238217157,0.029493726846163477,Original +1584,0.4406779661016949,0.3038135593220339,-9.848354363498629,0.0022258218489771527,Original +1585,0.3728813559322034,0.30819209039548023,-2.8185816226035203,0.06681666397008225,Original +1586,0.423728813559322,0.3077683615819209,-4.319796363784691,0.02285889788909435,Original +1587,0.36666666666666664,0.3008474576271187,-3.7673867088977024,0.032724190890523516,Original +1588,0.4406779661016949,0.32909604519774016,-5.006421579143373,0.015338372293344877,Original +1589,0.26666666666666666,0.1822033898305085,-10.409843826150722,0.001891887685589342,Original +1590,0.3728813559322034,0.32895480225988705,-2.949093002418226,0.060068491192589654,Original +1591,0.4576271186440678,0.4213747645951036,-3.60190807688241,0.06917625132966015,Original +1592,0.4067796610169492,0.3712570621468927,-2.1556241033270016,0.12006957075894573,Original +1593,0.3728813559322034,0.3081214689265537,-3.8998685439923104,0.029923600161968773,Original +1594,0.4067796610169492,0.3206214689265537,-3.0461946335774264,0.05559466924951487,Original +1595,0.5084745762711864,0.3803672316384181,-2.5550917784808758,0.08357084637244536,Original +1596,0.3898305084745763,0.3311676082862524,-2.2490974729241855,0.1534474328108915,Original +1597,0.4576271186440678,0.3293079096045198,-8.10248857720817,0.003929152121721815,Original +1598,0.36666666666666664,0.3432203389830508,-5.53333333333333,0.011632297103592594,Original +1599,0.4915254237288136,0.401271186440678,-2.459129645217466,0.09093785730195074,Original +1600,0.5423728813559322,0.4557909604519774,-3.177872547530921,0.05017596813649382,Original +1601,0.23728813559322035,0.19837570621468925,-4.553719008264464,0.01984639027416479,Original +1602,0.4745762711864407,0.39279661016949147,-2.2752115860184423,0.10740785666680364,Original +1603,0.4745762711864407,0.3667372881355932,-2.5590817041221827,0.08328072516772207,Original +1604,0.3728813559322034,0.307909604519774,-4.795831523312723,0.017248843874307026,Original +1605,0.4406779661016949,0.39646892655367233,-2.0743037811227554,0.12970694368573776,Original +1606,0.3728813559322034,0.29199623352165727,-4.218647880820993,0.05185732044957473,Original +1607,0.3728813559322034,0.307909604519774,-3.8877095717511785,0.030167484179746327,Original +1608,0.43333333333333335,0.3389830508474576,-3.1281966106828594,0.05213847018410804,Original +1609,0.423728813559322,0.2911016949152542,-3.8194407004959707,0.03158527352615677,Original +1610,0.36666666666666664,0.2796610169491525,-3.422222222222223,0.041774946157591517,Original +1611,0.35,0.3008474576271186,-3.396132253860897,0.042582247398141884,Original +1612,0.5254237288135594,0.4008474576271187,-5.126174517160044,0.01437396472653316,Original +1613,0.4745762711864407,0.4093926553672317,-3.8752447538675496,0.030420159550251232,Original +1614,0.4166666666666667,0.3220338983050848,-2.456339176607667,0.0911637428376164,Original +1615,0.4067796610169492,0.28721751412429375,-3.7810435820759127,0.03242035778750225,Original +1616,0.4067796610169492,0.3291666666666667,-4.334673395356118,0.022650355990863152,Original +1617,0.4745762711864407,0.2828389830508475,-10.598274792221071,0.0017948089853278408,Original +1618,0.4166666666666667,0.3050847457627119,-2.1744034361245617,0.1179670753099225,Original +1619,0.6271186440677966,0.4050847457627119,-9.240027567747635,0.0026818554468391116,Original +1620,0.4406779661016949,0.3543785310734463,-7.613153050047849,0.004703696031017581,Original +1621,0.45,0.3940677966101695,-2.976518270891203,0.05876051507230344,Original +1622,0.4745762711864407,0.38411016949152543,-7.506646894907283,0.0048984693502791745,Original +1623,0.4576271186440678,0.39668079096045195,-3.674203780886493,0.034897733800121254,Original +1624,0.4406779661016949,0.3627824858757062,-5.044529690751584,0.015022564694235147,Original +1625,0.4576271186440678,0.32083333333333336,-5.361374140743174,0.012698680109057153,Original +1626,0.4915254237288136,0.371045197740113,-5.171216742476558,0.014031731053501115,Original +1627,0.38333333333333336,0.3347457627118644,-3.822222222222226,0.03152585651673634,Original +1628,0.3389830508474576,0.3038135593220339,-27.666666666666476,0.00010364812386939461,Original +1629,0.4406779661016949,0.3425612052730697,-7.542807340088425,0.01712633052558875,Original +1630,0.423728813559322,0.2867937853107344,-3.780359197747885,0.032435497445757994,Original +1631,0.423728813559322,0.35430790960451974,-2.986076879787708,0.058313047921487345,Original +1632,0.3220338983050847,0.2911016949152543,-2.5053535141492107,0.08729389266327836,Original +1633,0.5084745762711864,0.38389830508474576,-3.5352476162925313,0.03849432965830513,Original +1634,0.4745762711864407,0.3296610169491525,-2.932194632545476,0.060892602743676055,Original +1635,0.3898305084745763,0.3145951035781544,-5.112873614515615,0.036189580961726336,Original +1636,0.4067796610169492,0.388135593220339,-3.316624790355395,0.04516783583455813,Original +1637,0.2711864406779661,0.24039548022598872,-2.2432542767385697,0.1106272525728738,Original +1638,0.4067796610169492,0.2913135593220339,-4.63515148563193,0.01891930032646319,Original +1639,0.423728813559322,0.31645480225988704,-7.447420720142193,0.005011398574053057,Original +1640,0.5,0.4025423728813559,-4.372697328618899,0.02212841807396774,Original +1641,0.4406779661016949,0.36525423728813555,-9.888888888888882,0.010071751888174055,Original +1642,0.4067796610169492,0.31631355932203387,-4.212334695138946,0.024441477588059136,Original +1643,0.4745762711864407,0.3838983050847458,-13.55263062255603,0.0008688656783751102,Original +1644,0.4915254237288136,0.29583333333333334,-3.9331245586533203,0.029269369284174996,Original +1645,0.4067796610169492,0.3418079096045198,-5.938574464184704,0.00954502881900154,Original +1646,0.5423728813559322,0.42189265536723164,-2.446859704851505,0.09193623834826871,Original +1647,0.4406779661016949,0.31242937853107344,-7.44762841439633,0.005010996583977287,Original +1648,0.423728813559322,0.35021186440677965,-6.955318569856883,0.006096907162548966,Original +1649,0.4576271186440678,0.3627824858757063,-6.8705764571632155,0.006314278455610205,Original +1650,0.3898305084745763,0.3165254237288136,-8.288366308708238,0.0036792632033313915,Original +1651,0.4067796610169492,0.31666666666666665,-4.566003911847601,0.019702811079520054,Original +1652,0.4166666666666667,0.364406779661017,-2.9623708453941098,0.05943073144700672,Original +1653,0.4067796610169492,0.3332627118644068,-2.9486299139697687,0.06009088857589412,Original +1654,0.3898305084745763,0.32040960451977407,-1.8464888902941,0.16198897032955312,Original +1655,0.423728813559322,0.341454802259887,-2.2689902462968896,0.10802564058574593,Original +1656,0.4406779661016949,0.31242937853107344,-5.102734357993427,0.01455637683815113,Original +1657,0.35,0.2754237288135593,-2.66340767777208,0.07611699066362566,Original +1658,0.4406779661016949,0.35035310734463276,-4.624808617471442,0.019033890844563105,Original +1659,0.4406779661016949,0.37549435028248584,-2.153106812065672,0.12035479878967444,Original +1660,0.2711864406779661,0.24708097928436912,-2.461538461538462,0.13291549663455224,Original +1661,0.3389830508474576,0.30798022598870056,-2.5144900114567874,0.08659496027337239,Original +1662,0.3898305084745763,0.26151129943502827,-8.20761557188874,0.0037851627982459913,Original +1663,0.4,0.3177966101694915,-3.377106525540539,0.043183492567890926,Original +1664,0.5084745762711864,0.39216101694915256,-4.374860785929049,0.02209919137309572,Original +1665,0.48333333333333334,0.3940677966101695,-4.0051372633147,0.027914510109473,Original +1666,0.423728813559322,0.30798022598870056,-5.52252684001195,0.011695784203535333,Original +1667,0.4576271186440678,0.41334745762711866,-3.0332358222638836,0.05616694738868672,Original +1668,0.4406779661016949,0.33326271186440676,-3.2380690490156394,0.04792113259547419,Original +1669,0.4067796610169492,0.3247175141242938,-3.864669010850272,0.030636676156145785,Original +1670,0.423728813559322,0.3754943502824859,-18.459459459459406,0.0003469320912607935,Original +1671,0.4067796610169492,0.35444915254237286,-7.394856051762754,0.005114520056169129,Original +1672,0.423728813559322,0.38389830508474576,-2.591193878173862,0.08099031623314684,Original +1673,0.423728813559322,0.3372175141242938,-2.2461944693225684,0.11032623843400814,Original +1674,0.4576271186440678,0.35444915254237286,-69.57142857142838,6.544177344803568e-06,Original +1675,0.4,0.3347457627118644,-2.262423733759691,0.10868236109185891,Original +1676,0.43333333333333335,0.3305084745762712,-2.886707772014044,0.06318231943174393,Original +1677,0.4745762711864407,0.39661016949152544,-5.396227020339848,0.012472526064506972,Original +1678,0.4067796610169492,0.3483050847457627,-11.775818566563489,0.00713429023847155,Original +1679,0.4406779661016949,0.3541666666666667,-3.8771841553769035,0.030380667708462643,Original +1680,0.423728813559322,0.3629237288135593,-4.044393553582212,0.02720985020655579,Original +1681,0.3728813559322034,0.30798022598870056,-2.584061622196982,0.08149225560483021,Original +1682,0.4745762711864407,0.3711158192090395,-5.036029713302375,0.015092265651518118,Original +1683,0.4745762711864407,0.38813559322033897,-3.0606121836612235,0.05496655926720403,Original +1684,0.5084745762711864,0.3965395480225989,-4.332720166230066,0.022677595074616603,Original +1685,0.4406779661016949,0.3933145009416196,-6.776908139806704,0.021087634455750442,Original +1686,0.4915254237288136,0.40896892655367234,-2.7252378965546162,0.07222977633186194,Original +1687,0.4576271186440678,0.35854519774011295,-7.385335801707823,0.005133495507004227,Original +1688,0.4406779661016949,0.3754943502824859,-5.31399624894294,0.013014777489807249,Original +1689,0.4067796610169492,0.32881355932203393,-3.1543372140892108,0.051093920277185956,Original +1690,0.4406779661016949,0.3628531073446328,-5.382081108767228,0.012563676575340528,Original +1691,0.43333333333333335,0.34745762711864403,-7.849246248313698,0.004307003389198978,Original +1692,0.4067796610169492,0.3163841807909604,-2.724020898427997,0.07230388107794879,Original +1693,0.3898305084745763,0.3038135593220339,-4.135972456516446,0.025652892626118445,Original +1694,0.3559322033898305,0.2786016949152542,-6.564422610165478,0.007190161439169192,Original +1695,0.45,0.36864406779661013,-3.3422909943494004,0.04431191289649832,Original +1696,0.5,0.364406779661017,-4.131182235954578,0.02573144168204107,Original +1697,0.5,0.3813559322033898,-2.919201796799047,0.06153589112407167,Original +1698,0.5084745762711864,0.35444915254237286,-4.447863008870363,0.021141608672485282,Original +1699,0.559322033898305,0.34611581920903955,-7.952423126063747,0.004147501211922912,Original +1700,0.423728813559322,0.354590395480226,-4.195847013586463,0.024696683385781026,Original +1701,0.5084745762711864,0.4514124293785311,-2.7684223456685544,0.06966019639221771,Original +1702,0.4666666666666667,0.38983050847457623,-3.3480885546378523,0.04412143039369844,Original +1703,0.3898305084745763,0.3332627118644068,-3.6621184863223024,0.03519301414612658,Original +1704,0.423728813559322,0.33742937853107347,-4.962234528821576,0.015715475861098616,Original +1705,0.4406779661016949,0.40098870056497177,-3.2752006464574146,0.04659397436436988,Original +1706,0.35,0.2923728813559322,-3.2984845004941277,0.04578545695456099,Original +1707,0.3333333333333333,0.2754237288135593,-3.609848715935058,0.03650753702324681,Original +1708,0.4666666666666667,0.3813559322033898,-10.066666666666663,0.0020873570938939115,Original +1709,0.4,0.3220338983050848,-5.039047529047529,0.01506747099332577,Original +1710,0.4,0.3432203389830508,-3.9231182932531077,0.029464267753910946,Original +1711,0.55,0.43644067796610164,-2.3596084289526735,0.09943521206177618,Original +1712,0.3559322033898305,0.3360169491525423,-1.4242424242424274,0.38970838325942464,Original +1713,0.5423728813559322,0.4132768361581921,-5.184819470451837,0.013930470217760019,Original +1714,0.4745762711864407,0.4091101694915254,-3.301274514960128,0.04568977195819744,Original +1715,0.4576271186440678,0.2870762711864407,-8.442559461787118,0.003487683310568225,Original +1716,0.4745762711864407,0.40508474576271186,-4.984211525609615,0.015526427125038359,Original +1717,0.4406779661016949,0.40084745762711865,-9.945373183606224,0.00216282978388815,Original +1718,0.4745762711864407,0.3882062146892655,-4.699364429896171,0.018227552356258447,Original +1719,0.3898305084745763,0.28700564971751413,-4.0757619541037435,0.02666317079369889,Original +1720,0.4067796610169492,0.358545197740113,-5.255556820568775,0.01341898591418548,Original +1721,0.5084745762711864,0.34187853107344635,-4.739270182928911,0.017814153415116545,Original +1722,0.4745762711864407,0.36299435028248583,-4.303404214552464,0.02309156259614923,Original +1723,0.4406779661016949,0.4049435028248588,-2.885264224008987,0.06325673205698319,Original +1724,0.3389830508474576,0.2825564971751412,-4.239397969490223,0.024029911324908407,Original +1725,0.3389830508474576,0.2996468926553672,-2.9888839714804116,0.058182452145839086,Original +1726,0.3898305084745763,0.2979284369114878,-5.611049007782942,0.030324925803336726,Original +1727,0.3728813559322034,0.3248587570621469,-6.42539604115687,0.0076408389421474725,Original +1728,0.4576271186440678,0.3161016949152543,-4.91247301430133,0.016154741914472536,Original +1729,0.4576271186440678,0.39668079096045195,-4.042742188321896,0.02723902806724627,Original +1730,0.3898305084745763,0.25748587570621473,-2.8937262538718946,0.06282208280557401,Original +1731,0.4067796610169492,0.30409604519774014,-3.6040922833026183,0.036656113662429206,Original +1732,0.4915254237288136,0.4051553672316384,-6.7314755578400405,0.006693743268842144,Original +1733,0.45,0.326271186440678,-8.548884639029154,0.00336320130822139,Original +1734,0.4067796610169492,0.2996468926553672,-5.609714558498771,0.011196176065376142,Original +1735,0.4406779661016949,0.37542372881355934,-3.3908313960680885,0.04274868888333008,Original +1736,0.35,0.3093220338983051,-3.1999999999999997,0.04933184296269623,Original +1737,0.4745762711864407,0.38201506591337103,-6.299599690538863,0.024284334261122562,Original +1738,0.4915254237288136,0.35021186440677965,-13.369445204883407,0.0009045905316411595,Original +1739,0.4,0.2923728813559322,-5.374733592842522,0.012611364632413758,Original +1740,0.4915254237288136,0.4219632768361582,-39.40000000000022,3.59729328035016e-05,Original +1741,0.559322033898305,0.3708333333333333,-4.916019118203146,0.016122913876139212,Original +1742,0.45,0.3008474576271187,-13.987061690606863,0.0007913286284483068,Original +1743,0.43333333333333335,0.3177966101694915,-4.258337907495993,0.023747193827929766,Original +1744,0.3389830508474576,0.2572033898305085,-4.033986241649274,0.027394411372010617,Original +1745,0.4745762711864407,0.3540489642184557,-3.3684210526315805,0.07796676388459127,Original +1746,0.4745762711864407,0.38813559322033897,-4.245089541976653,0.02394449803344903,Original +1747,0.4067796610169492,0.3204802259887006,-5.112836449747386,0.014477393996814832,Original +1748,0.5333333333333333,0.41949152542372886,-6.516123792642811,0.007342666934462457,Original +1749,0.4576271186440678,0.4093220338983051,-4.384615384615384,0.02196803265379433,Original +1750,0.4576271186440678,0.32902542372881355,-4.153671286484361,0.025365339369804084,Original +1751,0.4406779661016949,0.367090395480226,-3.028199837684228,0.05639133971192534,Original +1752,0.4576271186440678,0.3800847457627119,-2.6499484677906273,0.07699722400216412,Original +1753,0.4745762711864407,0.3713983050847458,-3.082793063741217,0.054017563157762766,Original +1754,0.3389830508474576,0.2866290018832392,-4.709835325623289,0.04224443000728903,Original +1755,0.3559322033898305,0.2532015065913371,-2.2513229059561555,0.15321033928811528,Original +1756,0.4406779661016949,0.3763653483992467,-3.2352232779643906,0.0837182578425499,Original +1757,0.3728813559322034,0.33742937853107347,-2.4658853737705697,0.09039381717094448,Original +1758,0.4406779661016949,0.35847457627118645,-5.143398239932954,0.014241825510816557,Original +1759,0.45,0.3813559322033898,-4.676537180435968,0.018469641646674063,Original +1760,0.423728813559322,0.363135593220339,-2.293068697718329,0.10565819843234399,Original +1761,0.4406779661016949,0.38375706214689265,-2.3710395141336553,0.09841141140216388,Original +1762,0.5254237288135594,0.42664783427495295,-3.434253416143983,0.07533225481738264,Original +1763,0.2542372881355932,0.2189265536723164,-2.272727272727272,0.15095548648584448,Original +1764,0.43333333333333335,0.3940677966101695,-1.6131196118561335,0.20511808054337077,Original +1765,0.5254237288135594,0.4224576271186441,-2.122911802712327,0.1238400486795913,Original +1766,0.48333333333333334,0.3983050847457627,-3.4756486205215467,0.04018159901970025,Original +1767,0.48333333333333334,0.4067796610169492,-3.4985949560740326,0.03952103415356299,Original +1768,0.3559322033898305,0.23634651600753295,-3.432432432432432,0.07540339116695642,Original +1769,0.4576271186440678,0.3125,-3.6292186583582198,0.03601320371125468,Original +1770,0.4406779661016949,0.38757062146892657,-6.714285714285707,0.021470189769271106,Original +1771,0.4166666666666667,0.30508474576271183,-5.099428072506434,0.01458234892459302,Original +1772,0.38333333333333336,0.3432203389830508,-4.943805429141822,0.015876322370823124,Original +1773,0.423728813559322,0.28255649717514125,-7.881999450070924,0.004255505421232205,Original +1774,0.4915254237288136,0.40494350282485875,-5.023746418203985,0.015193736814334122,Original +1775,0.559322033898305,0.4595338983050848,-2.392754433686525,0.09650164759323027,Original +1776,0.4745762711864407,0.35459039548022603,-5.888771927947271,0.009773561677083785,Original +1777,0.4406779661016949,0.32478813559322034,-12.454354190804333,0.0011156250386489563,Original +1778,0.5084745762711864,0.3755649717514124,-3.3684353047674622,0.04346109463220811,Original +1779,0.423728813559322,0.3672316384180791,-4.082482904638627,0.026547885467199526,Original +1780,0.4576271186440678,0.35402542372881357,-2.5536843901385566,0.08367347896165804,Original +1781,0.4745762711864407,0.4175141242937853,-1.946993604286342,0.1467043534415175,Original +1782,0.4576271186440678,0.37966101694915255,-3.993713653195303,0.028123969733242882,Original +1783,0.4576271186440678,0.354590395480226,-2.8307905599569243,0.06614653353035789,Original +1784,0.3898305084745763,0.32090395480225986,-3.177686792530804,0.05018313128134387,Original +1785,0.5084745762711864,0.4304378531073446,-6.772008114046649,0.006580141599024013,Original +1786,0.4067796610169492,0.3839689265536724,-5.701874626813004,0.010698008944756929,Original +1787,0.5254237288135594,0.3542372881355932,-5.854047900359961,0.009937162378535997,Original +1788,0.5084745762711864,0.3412429378531074,-3.8779880896871726,0.030364316576825364,Original +1789,0.5084745762711864,0.42196327683615825,-3.458948005905017,0.04067122554425211,Original +1790,0.4166666666666667,0.3135593220338983,-1.8272855362744203,0.16511600726806364,Original +1791,0.4406779661016949,0.3377118644067797,-6.269358747755155,0.00819196116993917,Original +1792,0.5932203389830508,0.40939265536723163,-13.436639703627074,0.0008912634414880094,Original +1793,0.3898305084745763,0.2910310734463277,-4.146372958429369,0.025483409105270218,Original +1794,0.3728813559322034,0.3313559322033898,-1.9155635193164227,0.19549520636629938,Original +1795,0.4745762711864407,0.38728813559322034,-2.7837837837837838,0.10845063612547547,Original +1796,0.4915254237288136,0.321045197740113,-5.483588555093928,0.011928337117442485,Original +1797,0.4406779661016949,0.3612994350282486,-14.789473684210558,0.042980044870241256,Original +1798,0.5084745762711864,0.3962570621468926,-3.830072253349244,0.0313589410287926,Original +1799,0.4406779661016949,0.3246468926553672,-3.659896369736786,0.035247654519723955,Original +1800,0.3389830508474576,0.2919962335216572,-3.5407885315000662,0.07133333882574665,Original +1801,0.423728813559322,0.3584039548022599,-3.1685329501250483,0.050537730061605154,Original +1802,0.4067796610169492,0.32076271186440675,-6.808401479691212,0.006480293784839257,Original +1803,0.3898305084745763,0.34625706214689267,-2.0407541348149305,0.13394942774558222,Original +1804,0.4666666666666667,0.3516949152542373,-5.158515399269249,0.014127150588176329,Original +1805,0.4745762711864407,0.37146892655367225,-5.367066741284801,0.012661377799601295,Original +1806,0.4915254237288136,0.3967514124293785,-2.4964946840644773,0.08797820274589993,Original +1807,0.5333333333333333,0.4745762711864407,-2.3551362310207575,0.09983927564004276,Original +1808,0.4745762711864407,0.4009180790960452,-8.122451922976826,0.003901259023001135,Original +1809,0.4745762711864407,0.3456920903954802,-4.094924639556887,0.02633616071520719,Original +1810,0.4745762711864407,0.3463276836158192,-3.5851541120965997,0.03715039979658868,Original +1811,0.3898305084745763,0.3125,-2.848001248439178,0.06521603391989962,Original +1812,0.4067796610169492,0.2955508474576271,-5.314610961015995,0.013010610801463965,Original +1813,0.4915254237288136,0.3711864406779661,-6.051234040593998,0.009053328678904286,Original +1814,0.3559322033898305,0.2994350282485876,-2.6261286571944504,0.07858610153137385,Original +1815,0.4166666666666667,0.35593220338983056,-3.925344995453686,0.02942075287920854,Original +1816,0.5254237288135594,0.43481638418079094,-4.495559790237215,0.02054484962621015,Original +1817,0.3898305084745763,0.3416666666666667,-5.211728279194447,0.01373294914761014,Original +1818,0.5254237288135594,0.4387711864406779,-3.071254985149466,0.05450861721232613,Original +1819,0.4915254237288136,0.3923022598870056,-2.724050147464563,0.07230209895638379,Original +1820,0.576271186440678,0.40494350282485875,-6.15102387874345,0.008645202687168257,Original +1821,0.4067796610169492,0.3374293785310734,-2.6487965059751333,0.07707314575519472,Original +1822,0.423728813559322,0.2919962335216572,-9.926980271680543,0.009995759216947474,Original +1823,0.4745762711864407,0.3204802259887006,-6.442102133047244,0.007584755136376526,Original +1824,0.5,0.4279661016949153,-4.9770903720375195,0.015587358121496146,Original +1825,0.5333333333333333,0.3432203389830508,-10.881765043296808,0.0016608270081661155,Original +1826,0.4576271186440678,0.38799435028248586,-1.7579248960751477,0.17700539897601802,Original +1827,0.4915254237288136,0.36271186440677966,-3.4834542887072035,0.03995532706040686,Original +1828,0.4745762711864407,0.40932203389830507,-12.89316271749542,0.0010070852904372337,Original +1829,0.423728813559322,0.3586158192090395,-5.301653087443131,0.013098812973106142,Original +1830,0.45,0.41101694915254233,-2.6934842013379563,0.07419455966723891,Original +1831,0.423728813559322,0.36694915254237287,-2.416081794014973,0.0945001250957143,Original +1832,0.4576271186440678,0.35028248587570626,-4.358898943540671,0.02231600572520053,Original +1833,0.4067796610169492,0.35882768361581924,-2.596202410283425,0.08064011223189176,Original +1834,0.4406779661016949,0.3968220338983051,-2.5012480459007254,0.08761020945997577,Original +1835,0.4745762711864407,0.40918079096045196,-3.6547443425101043,0.03537475739142397,Original +1836,0.5084745762711864,0.4093220338983051,-6.728880964155069,0.006701102825924804,Original +1837,0.4406779661016949,0.3880649717514124,-3.6962350907584107,0.03436754879976233,Original +1838,0.43333333333333335,0.3135593220338983,-3.0363270659271957,0.056029764059456005,Original +1839,0.5254237288135594,0.3624293785310735,-4.007620942190811,0.02786923613375584,Original +1840,0.3898305084745763,0.3442090395480226,-1.1660649819494582,0.4512877748622783,Original +1841,0.4576271186440678,0.36299435028248583,-5.564047150200577,0.011454300879796516,Original +1842,0.45,0.35593220338983045,-4.53155602414888,0.020108869526374775,Original +1843,0.5,0.3728813559322034,-3.2995600879804483,0.04574853906846063,Original +1844,0.4745762711864407,0.3801553672316384,-2.6218256817858103,0.0788774316850411,Original +1845,0.4067796610169492,0.34625706214689267,-2.2952786442989725,0.10544407020716878,Original +1846,0.4576271186440678,0.34194915254237285,-7.414573731136393,0.005075512686812491,Original +1847,0.4406779661016949,0.29540960451977405,-4.994116161502297,0.015442194581693753,Original +1848,0.3389830508474576,0.2660310734463277,-4.308319914295902,0.02302147063382807,Original +1849,0.5254237288135594,0.392090395480226,-3.99713035331454,0.028061110834153927,Original +1850,0.4166666666666667,0.34745762711864403,-3.430686205868627,0.04151723186686211,Original +1851,0.3898305084745763,0.35854519774011295,-2.331934255279095,0.10196787590396043,Original +1852,0.4576271186440678,0.2998587570621469,-5.194000650735758,0.013862662574053051,Original +1853,0.4576271186440678,0.2990819209039548,-3.51802668500397,0.038972429516855533,Original +1854,0.4067796610169492,0.34173728813559323,-2.5913248074535873,0.08098113762773655,Original +1855,0.43333333333333335,0.3347457627118644,-5.6429955425120175,0.011012841275964533,Original +1856,0.4915254237288136,0.3794491525423729,-3.26894241254243,0.04681436566609524,Original +1857,0.5423728813559322,0.40084745762711865,-3.985615868631576,0.028273674351328677,Original +1858,0.4406779661016949,0.34597457627118644,-6.54645011829731,0.007246419253764158,Original +1859,0.423728813559322,0.2955508474576271,-3.1245537578699243,0.05228616491106145,Original +1860,0.423728813559322,0.2823446327683616,-3.3366481297841037,0.04449831598300532,Original +1861,0.4666666666666667,0.3898305084745763,-4.966017854713506,0.015682718436889013,Original +1862,0.4915254237288136,0.3248587570621469,-3.666081619407081,0.03509583190764937,Original +1863,0.4406779661016949,0.35466101694915253,-3.9441648534374134,0.029056249071256115,Original +1864,0.43333333333333335,0.36864406779661013,-1.786828180527595,0.17193506795662192,Original +1865,0.6440677966101694,0.4304378531073446,-11.862745098039216,0.001287998117223525,Original +1866,0.576271186440678,0.43036723163841806,-3.707083667393408,0.034110263406932406,Original +1867,0.5084745762711864,0.379590395480226,-3.550388805186171,0.03808013155197999,Original +1868,0.35,0.2838983050847458,-5.199999999999993,0.013818586905132142,Original +1869,0.4576271186440678,0.3840395480225988,-3.832436694700159,0.03130888812484261,Original +1870,0.4406779661016949,0.3586864406779661,-3.3503877475316646,0.04404617677686765,Original +1871,0.5423728813559322,0.39244350282485874,-9.21746279562317,0.0027010591887466347,Original +1872,0.4406779661016949,0.3964689265536723,-2.3349757965803564,0.10168572245232489,Original +1873,0.4745762711864407,0.40911016949152545,-2.509542503119258,0.08697258350325465,Original +1874,0.423728813559322,0.3287429378531073,-2.9086605723710677,0.06206405682692753,Original +1875,0.4745762711864407,0.3425141242937853,-2.187920293054828,0.11648102879543593,Original +1876,0.4406779661016949,0.36694915254237287,-2.2961086998148015,0.10536377934120532,Original +1877,0.45,0.4152542372881356,-3.175846343890085,0.05025417308948302,Original +1878,0.3559322033898305,0.29971751412429376,-3.099929158257806,0.05329849219831802,Original +1879,0.4666666666666667,0.3305084745762712,-6.072629199681584,0.008963729862100802,Original +1880,0.3898305084745763,0.31101694915254235,-7.153846153846161,0.08841694696574594,Original +1881,0.4067796610169492,0.3038135593220339,-7.40913346851445,0.005086235812941151,Original +1882,0.4666666666666667,0.4152542372881356,-2.2929844695893116,0.10566636994498525,Original +1883,0.5423728813559322,0.37521186440677967,-4.801612169541382,0.017192389686773397,Original +1884,0.4406779661016949,0.33785310734463275,-3.9583928774422366,0.028784526660768115,Original +1885,0.4745762711864407,0.3879237288135593,-2.7149672381577274,0.07285814285484306,Original +1886,0.5833333333333334,0.38983050847457623,-7.2205339907178026,0.005477206760423499,Original +1887,0.4166666666666667,0.3686440677966102,-5.918640302493723,0.009635652367993155,Original +1888,0.423728813559322,0.3987758945386064,-1.8820851842053508,0.20054056943775883,Original +1889,0.4915254237288136,0.43015536723163844,-2.973639372537061,0.05889612816583925,Original +1890,0.5254237288135594,0.33361581920903954,-4.658888265058127,0.018659682503791214,Original +1891,0.4915254237288136,0.37097457627118646,-3.2449363634867825,0.04767210916456234,Original +1892,0.43333333333333335,0.36016949152542377,-2.612964350541548,0.07948159398225181,Original +1893,0.4576271186440678,0.32937853107344633,-4.435658555321488,0.021297902760540714,Original +1894,0.423728813559322,0.3543785310734463,-7.723907549595242,0.004511849154258445,Original +1895,0.4915254237288136,0.41765536723163843,-6.264933372064462,0.008208341834463534,Original +1896,0.4406779661016949,0.3501412429378531,-4.144463927530725,0.02551440964648421,Original +1897,0.423728813559322,0.30798022598870056,-8.186715143005392,0.003813227168001511,Original +1898,0.4406779661016949,0.3123587570621469,-4.795397246883075,0.017253094776815005,Original +1899,0.5333333333333333,0.40254237288135597,-3.012278225167804,0.0571082206033358,Original +1900,0.4,0.3008474576271186,-3.099405715532204,0.053320277974167773,Original +1901,0.4067796610169492,0.29562146892655367,-3.85816763155035,0.030770761179515826,Original +1902,0.4745762711864407,0.3714689265536723,-4.782389009187055,0.017381059516950674,Original +1903,0.4166666666666667,0.3220338983050848,-3.7931280643843954,0.03215451606094398,Original +1904,0.4406779661016949,0.40084745762711865,-2.4921620337819483,0.08831527775786965,Original +1905,0.4915254237288136,0.4009180790960452,-9.991472499692488,0.0021337257653321886,Original +1906,0.36666666666666664,0.3220338983050848,-4.561067126598043,0.019760347058840888,Original +1907,0.5254237288135594,0.4262005649717514,-6.726801374615161,0.006707009265801296,Original +1908,0.4745762711864407,0.34611581920903955,-7.571218430433462,0.004779142305515517,Original +1909,0.3898305084745763,0.286864406779661,-11.301306940901064,0.0014858566277749805,Original +1910,0.4576271186440678,0.32076271186440675,-8.564496343017598,0.0033454209602583485,Original +1911,0.4745762711864407,0.3628531073446328,-4.591316366529474,0.019411195178700307,Original +1912,0.4576271186440678,0.3208333333333333,-10.424077229245048,0.0018843150010148248,Original +1913,0.45,0.30508474576271183,-3.4430305632479454,0.04114498062149175,Original +1914,0.4067796610169492,0.36709039548022593,-9.92864680939292,0.0021735197350935946,Original +1915,0.5333333333333333,0.3305084745762712,-41.4537493278151,3.0893720674523154e-05,Original +1916,0.5423728813559322,0.33785310734463275,-7.8732869320554375,0.004269124399355101,Original +1917,0.5932203389830508,0.38834745762711864,-10.638370665053376,0.0017750090525035277,Original +1918,0.4,0.3389830508474576,-3.9436024140371986,0.029067057994347754,Original +1919,0.4406779661016949,0.3209039548022599,-5.521881967348604,0.011699587084326106,Original +1920,0.5084745762711864,0.29964689265536726,-5.367242638173358,0.01266022746944358,Original +1921,0.4576271186440678,0.3714689265536723,-5.21158171207224,0.013734015073651634,Original +1922,0.4406779661016949,0.35035310734463276,-4.9456020241512695,0.015860547909964277,Original +1923,0.3728813559322034,0.2750470809792844,-4.254446720326273,0.05105369953469059,Original +1924,0.4067796610169492,0.34597457627118644,-14.428366472211776,0.0007217047864811893,Original +1925,0.5084745762711864,0.3921610169491525,-5.123685756910068,0.014393190365221242,Original +1926,0.4406779661016949,0.3547316384180791,-2.6999881415740097,0.07378677879364155,Original +1927,0.4576271186440678,0.31221751412429377,-4.544899514360726,0.01995030538814641,Original +1928,0.4406779661016949,0.34246704331450095,-4.8064516129032215,0.04066429179543451,Original +1929,0.559322033898305,0.3800847457627119,-3.055206987855407,0.05520098921136872,Original +1930,0.35,0.2754237288135593,-2.66340767777208,0.07611699066362566,Original +1931,0.5254237288135594,0.4429378531073446,-6.248223190659335,0.008270584465932308,Original +1932,0.3898305084745763,0.2992231638418079,-3.1375946196106543,0.05175986194673748,Original +1933,0.4576271186440678,0.33771186440677964,-7.301351957262182,0.005304981348263639,Original +1934,0.4067796610169492,0.3080508474576271,-3.1525851210756604,0.051163101927519376,Original +1935,0.3389830508474576,0.29124293785310734,-3.5699799446212253,0.03755260022285919,Original +1936,0.423728813559322,0.38001412429378534,-2.034149230295387,0.13480390529326824,Original +1937,0.3898305084745763,0.3374293785310734,-2.3577069273967326,0.09960676963753387,Original +1938,0.4576271186440678,0.3080508474576271,-17.7463880302814,0.0003901203293975093,Original +1939,0.4576271186440678,0.3704331450094162,-1.7365199955826325,0.22460571149425088,Original +1940,0.4406779661016949,0.328954802259887,-5.202033883431729,0.013803686006307778,Original +1941,0.4745762711864407,0.3628531073446327,-3.2057938752604676,0.04911378606623852,Original +1942,0.4745762711864407,0.35444915254237286,-3.7842912965837407,0.032348636596183485,Original +1943,0.5932203389830508,0.30007062146892655,-5.840104693319583,0.010003865123379863,Original +1944,0.4745762711864407,0.3880414312617702,-2.3384829558815743,0.14430805754256285,Original +1945,0.4067796610169492,0.37071563088512244,-2.2572802622847377,0.15257812860009795,Original +1946,0.3898305084745763,0.358545197740113,-2.3319342552790907,0.10196787590396081,Original +1947,0.423728813559322,0.3627824858757062,-1.9704993917573732,0.1433770749616963,Original +1948,0.4666666666666667,0.3813559322033898,-2.4415252920324195,0.09237446595146682,Original +1949,0.4915254237288136,0.3713276836158192,-6.5420904956073125,0.007260153005952,Original +1950,0.4406779661016949,0.3334745762711865,-3.470097292282929,0.04034351536878077,Original +1951,0.4745762711864407,0.3879237288135593,-2.1390171916328864,0.12196630264944257,Original +1952,0.48333333333333334,0.40254237288135597,-2.8010960513215193,0.06779122416384437,Original +1953,0.423728813559322,0.379590395480226,-2.325606550774779,0.10255792463406842,Original +1954,0.4915254237288136,0.40433145009416194,-2.662987333476817,0.11681579707766619,Original +1955,0.4915254237288136,0.3373587570621469,-5.418875755811864,0.012328378409982957,Original +1956,0.4406779661016949,0.32902542372881355,-5.097322089843262,0.014598923701575601,Original +1957,0.4745762711864407,0.38375706214689265,-2.795942882520684,0.06808180891277789,Original +1958,0.4406779661016949,0.3250706214689265,-4.171340373881652,0.02508240388592301,Original +1959,0.4915254237288136,0.3246468926553673,-6.228587468296307,0.008344518944513782,Original +1960,0.5,0.2838983050847458,-8.87796045374059,0.0030132990718159855,Original +1961,0.3898305084745763,0.2824858757062147,-4.417410272265133,0.021534399027710104,Original +1962,0.5084745762711864,0.39279661016949147,-3.218304471414686,0.04864708009336623,Original +1963,0.4406779661016949,0.3543785310734463,-3.815695684869742,0.03166549916342224,Original +1964,0.4406779661016949,0.31228813559322033,-11.326278432886252,0.0014762298056976652,Original +1965,0.4576271186440678,0.4092514124293785,-3.953202519249618,0.028883270408863487,Original +1966,0.423728813559322,0.36264124293785305,-2.925487770616238,0.06122360955751544,Original +1967,0.4576271186440678,0.359322033898305,-1.8552679320224146,0.20470801229945912,Original +1968,0.5254237288135594,0.3965395480225989,-9.383684565305357,0.002563738947888246,Original +1969,0.45,0.3474576271186441,-5.812651983124662,0.010136920657992399,Original +1970,0.4067796610169492,0.3081214689265537,-3.056600083912935,0.05514044883654204,Original +1971,0.4406779661016949,0.3456920903954802,-2.7626457956106245,0.06999725904230607,Original +1972,0.4067796610169492,0.3418079096045198,-4.4263520637871325,0.021418091531717674,Original +1973,0.5423728813559322,0.37528248587570623,-8.13082423873902,0.0038896389696946266,Original +1974,0.45,0.3050847457627119,-8.55,0.0033619268709983345,Original +1975,0.4406779661016949,0.3840395480225989,-6.277129309215832,0.00816330194750534,Original +1976,0.4166666666666667,0.3771186440677966,-1.974967724404075,0.14275464845123892,Original +1977,0.4406779661016949,0.36730225988700566,-2.6278882027424713,0.07846735545649752,Original +1978,0.6440677966101694,0.4809322033898305,-5.744562646538028,0.010477088755276323,Original +1979,0.4576271186440678,0.4137005649717514,-2.715044307099425,0.0728534025239453,Original +1980,0.559322033898305,0.2867937853107345,-3.7484025543573445,0.03315263271031142,Original +1981,0.4745762711864407,0.3584039548022599,-3.4358556656684405,0.041360824576078994,Original +1982,0.423728813559322,0.32895480225988705,-3.853459154520629,0.03086833989707754,Original +1983,0.43333333333333335,0.3135593220338983,-4.2613603366990604,0.0237024761862663,Original +1984,0.6101694915254238,0.4471751412429379,-10.661446386757238,0.0017637451315993058,Original +1985,0.3898305084745763,0.354590395480226,-2.6581313276202945,0.07646057355190569,Original +1986,0.5166666666666667,0.42372881355932207,-2.2081034216541404,0.11430368101220882,Original +1987,0.4406779661016949,0.3291666666666667,-4.203856330713102,0.024572283074481203,Original +1988,0.5254237288135594,0.400635593220339,-6.380724773314517,0.0077935062804952064,Original +1989,0.5084745762711864,0.35861581920903957,-2.999030425264594,0.05771345344001608,Original +1990,0.3728813559322034,0.295409604519774,-3.600128581629112,0.03675886811551069,Original +1991,0.559322033898305,0.40508474576271186,-8.389086394729976,0.003552594693743783,Original +1992,0.4745762711864407,0.3540254237288135,-2.4793636360304507,0.08932026186301084,Original +1993,0.4915254237288136,0.42189265536723164,-3.2416676813361778,0.04779043382080003,Original +1994,0.4406779661016949,0.3673728813559322,-2.8797359104311173,0.06354272267077887,Original +1995,0.4,0.3474576271186441,-2.978383660774621,0.05867285324441682,Original +1996,0.4406779661016949,0.3712570621468927,-2.386590270664442,0.09703914637860833,Original +1997,0.4745762711864407,0.4093926553672317,-2.226010109872328,0.11241274845462867,Original +1998,0.4406779661016949,0.38834745762711864,-3.1550201885686455,0.051066984905793825,Original +1999,0.5423728813559322,0.37097457627118646,-5.074685831579439,0.014778639324226026,Original +2000,0.48333333333333334,0.22814207650273222,-21.334991021776784,0.00022530414214046572,Watermarked +2001,0.4098360655737705,0.23333333333333334,-4.992251154606664,0.015458009685690827,Watermarked +2002,0.65,0.34815573770491803,-6.977885499593617,0.0060406875581721555,Watermarked +2003,0.6333333333333333,0.2573087431693989,-17.794177111160675,0.0003870090924213516,Watermarked +2004,0.4,0.3151639344262295,-1.713189822924711,0.18519433572899746,Watermarked +2005,0.45901639344262296,0.20416666666666666,-8.101361023294555,0.003930735409185079,Watermarked +2006,0.4166666666666667,0.27424863387978143,-2.647512144273123,0.07715790266759627,Watermarked +2007,0.5333333333333333,0.28599726775956286,-9.817142706536112,0.002246603044354501,Watermarked +2008,0.5245901639344263,0.2375,-5.570367388129549,0.011418116056075428,Watermarked +2009,0.4666666666666667,0.2775273224043716,-4.727075685541707,0.01793917650756737,Watermarked +2010,0.4098360655737705,0.25,-3.714254520543179,0.033941551397426564,Watermarked +2011,0.6333333333333333,0.24904371584699453,-14.463091326070671,0.0007165776065197027,Watermarked +2012,0.5081967213114754,0.275,-5.83498532451519,0.01002850287511932,Watermarked +2013,0.5245901639344263,0.23750000000000002,-15.536893060799459,0.0005793474370025991,Watermarked +2014,0.7,0.32745901639344266,-6.77625507348341,0.006568385846444286,Watermarked +2015,0.4426229508196721,0.21666666666666667,-4.119026835630454,0.025932160329463834,Watermarked +2016,0.6721311475409836,0.23333333333333334,-9.613578441019637,0.0023886490069146135,Watermarked +2017,0.38333333333333336,0.2775956284153005,-2.196141651943659,0.11558815206376069,Watermarked +2018,0.43333333333333335,0.29480874316939887,-3.9127157656292244,0.029668656491470317,Watermarked +2019,0.3770491803278688,0.2125,-4.70897478231848,0.018126865049367218,Watermarked +2020,0.6166666666666667,0.3523907103825137,-5.470026246143757,0.012010754748633298,Watermarked +2021,0.3442622950819672,0.2875,-2.0013563154719005,0.13914298161809877,Watermarked +2022,0.35,0.22786885245901642,-2.9204148617045544,0.06147547219131401,Watermarked +2023,0.43333333333333335,0.2573770491803279,-6.707544913199492,0.0067620282205820385,Watermarked +2024,0.6229508196721312,0.20833333333333334,-7.9670319147285165,0.004125551420928103,Watermarked +2025,0.3770491803278688,0.24583333333333332,-12.513573727485,0.0011000978118262336,Watermarked +2026,0.48333333333333334,0.31953551912568307,-7.4329764222856545,0.005039460040419282,Watermarked +2027,0.4918032786885246,0.2,-10.721537070870632,0.0017348546540608164,Watermarked +2028,0.45,0.1825136612021858,-9.313256255544996,0.002620765891462303,Watermarked +2029,0.45,0.27759562841530055,-4.406020658142731,0.02168373286648725,Watermarked +2030,0.5333333333333333,0.21967213114754097,-6.698633516057464,0.006787690665758875,Watermarked +2031,0.45,0.2444672131147541,-5.639134297794538,0.011033909197241593,Watermarked +2032,0.4,0.23224043715846998,-9.669790958355271,0.0023482624188435656,Watermarked +2033,0.5,0.21530054644808744,-4.776258392255463,0.017441795571057156,Watermarked +2034,0.5737704918032787,0.13749999999999998,-41.60551556348084,3.055734072793683e-05,Watermarked +2035,0.5245901639344263,0.19583333333333333,-11.940183637404086,0.001263509863921225,Watermarked +2036,0.36666666666666664,0.29439890710382516,-2.2491233682903635,0.11002736160816107,Watermarked +2037,0.55,0.21598360655737703,-12.468162549596142,0.0011119786230779946,Watermarked +2038,0.5409836065573771,0.27083333333333337,-8.587746675724997,0.0033191706279838665,Watermarked +2039,0.45,0.26584699453551913,-3.724628638572125,0.03369936057429459,Watermarked +2040,0.5166666666666667,0.34460382513661203,-3.314218356932012,0.0452491572848592,Watermarked +2041,0.5573770491803278,0.20416666666666666,-12.828316972577708,0.001022214116280989,Watermarked +2042,0.4166666666666667,0.27759562841530055,-4.08198313178942,0.026556436001686043,Watermarked +2043,0.6229508196721312,0.21250000000000002,-32.83606557377048,6.20825070326001e-05,Watermarked +2044,0.43333333333333335,0.24849726775956282,-3.748296362699404,0.03315504943587395,Watermarked +2045,0.5245901639344263,0.2625,-10.53248144497122,0.0018279382457190715,Watermarked +2046,0.4426229508196721,0.29166666666666663,-2.5878220140515227,0.0812271381568774,Watermarked +2047,0.5166666666666667,0.21598360655737703,-8.750839688124422,0.003142480189931068,Watermarked +2048,0.6333333333333333,0.232103825136612,-8.35174130824408,0.0035988783002935975,Watermarked +2049,0.5901639344262295,0.22083333333333333,-13.413790344368145,0.0008957658359722933,Watermarked +2050,0.5,0.27814207650273226,-3.820785157614083,0.03155653621002948,Watermarked +2051,0.7,0.20744535519125684,-28.591452014534944,9.39404974108921e-05,Watermarked +2052,0.39344262295081966,0.29583333333333334,-9.308639696291548,0.0026245624151365297,Watermarked +2053,0.45,0.20348360655737702,-5.6752883391613915,0.010838689050252247,Watermarked +2054,0.35,0.26939890710382514,-2.481709453531588,0.08913501383686977,Watermarked +2055,0.45,0.28592896174863386,-3.800322116899045,0.031997583361784786,Watermarked +2056,0.48333333333333334,0.2533469945355191,-6.384490208675115,0.007780483735954091,Watermarked +2057,0.4426229508196721,0.3333333333333333,-5.678855106783206,0.010819675519646264,Watermarked +2058,0.38333333333333336,0.26536885245901637,-2.8854448330676328,0.06324741595265697,Watermarked +2059,0.4918032786885246,0.2583333333333333,-6.9322595553517825,0.006155066763755107,Watermarked +2060,0.4426229508196721,0.3125,-7.574268290069089,0.004773601555369254,Watermarked +2061,0.6333333333333333,0.27377049180327867,-36.553205244976375,4.503243730199633e-05,Watermarked +2062,0.6,0.14508196721311475,-16.724450142912833,0.00046542921319575733,Watermarked +2063,0.6721311475409836,0.2791666666666667,-14.728977904018867,0.000678879499435165,Watermarked +2064,0.4166666666666667,0.30293715846994534,-4.392324491038346,0.021865089257387872,Watermarked +2065,0.4918032786885246,0.26249999999999996,-2.9860360155946784,0.05831495178179603,Watermarked +2066,0.5666666666666667,0.18237704918032788,-16.083566531034208,0.0005227722151560427,Watermarked +2067,0.45,0.24002732240437158,-3.3909481332454887,0.04274501456307789,Watermarked +2068,0.35,0.25703551912568307,-2.6079704990661914,0.07982458937806818,Watermarked +2069,0.38333333333333336,0.2782103825136612,-3.2919830028303934,0.04600941654903949,Watermarked +2070,0.5,0.27342896174863385,-3.9770819524294616,0.02843255328181686,Watermarked +2071,0.6721311475409836,0.22083333333333333,-8.141189027892935,0.0038753168087557023,Watermarked +2072,0.45901639344262296,0.27499999999999997,-7.102306152917264,0.005742682031524343,Watermarked +2073,0.45,0.21974043715846997,-4.850332934764336,0.016725995889598343,Watermarked +2074,0.4166666666666667,0.307103825136612,-12.422543126716244,0.0011240861310550566,Watermarked +2075,0.7049180327868853,0.19583333333333333,-12.238447225159446,0.001174764152702687,Watermarked +2076,0.45901639344262296,0.2416666666666667,-4.10684476131458,0.026135349573198702,Watermarked +2077,0.6666666666666666,0.21591530054644809,-21.638546889559418,0.00021600181591014574,Watermarked +2078,0.39344262295081966,0.25416666666666665,-3.491255808134439,0.0397307943380083,Watermarked +2079,0.38333333333333336,0.24849726775956282,-2.5061607287884398,0.08723186353716599,Watermarked +2080,0.45901639344262296,0.24166666666666667,-5.438466110898458,0.012205447523675413,Watermarked +2081,0.36065573770491804,0.275,-1.7374154679173113,0.18070726370520965,Watermarked +2082,0.5166666666666667,0.25669398907103824,-3.5751586418022976,0.03741471383610712,Watermarked +2083,0.6721311475409836,0.2791666666666667,-9.089163278771835,0.002813782076578305,Watermarked +2084,0.45901639344262296,0.2708333333333333,-5.018214936247723,0.01523972222476046,Watermarked +2085,0.6557377049180327,0.23333333333333334,-19.631581158153004,0.00028877725744096686,Watermarked +2086,0.43333333333333335,0.26133879781420766,-3.406748909038209,0.04225136416729629,Watermarked +2087,0.36666666666666664,0.22370218579234974,-3.560576108205079,0.03780464605083627,Watermarked +2088,0.6,0.29077868852459016,-8.670001234457226,0.003228460547141703,Watermarked +2089,0.6721311475409836,0.2791666666666667,-11.945500838297065,0.001261851176283919,Watermarked +2090,0.4166666666666667,0.2780054644808743,-7.059001645570319,0.005844158386152648,Watermarked +2091,0.6885245901639344,0.275,-14.961882623555587,0.0006479949161931086,Watermarked +2092,0.5081967213114754,0.24583333333333335,-10.170122389206956,0.002025718899581995,Watermarked +2093,0.5737704918032787,0.22916666666666669,-18.649446940693135,0.0003365083085818159,Watermarked +2094,0.45,0.24890710382513662,-17.77843472912827,0.0003880303178033284,Watermarked +2095,0.5573770491803278,0.2,-11.19804462246161,0.001526562128286031,Watermarked +2096,0.4,0.21154371584699455,-6.028584041580653,0.009149467578500266,Watermarked +2097,0.39344262295081966,0.2375,-5.077417805154715,0.014756796916773422,Watermarked +2098,0.43333333333333335,0.2695355191256831,-4.505004550324794,0.02042928010369098,Watermarked +2099,0.48333333333333334,0.30327868852459017,-5.139479033055133,0.014271753686067527,Watermarked +2100,0.5666666666666667,0.23162568306010928,-5.174508393880651,0.01400713962990326,Watermarked +2101,0.3770491803278688,0.2333333333333333,-2.6611003960675528,0.07626699967069282,Watermarked +2102,0.4918032786885246,0.2375,-8.083990107136325,0.003955234173311845,Watermarked +2103,0.5409836065573771,0.16666666666666669,-20.79300879817042,0.00024328442858722197,Watermarked +2104,0.5737704918032787,0.22916666666666663,-7.437123752141218,0.0050313817019966775,Watermarked +2105,0.5901639344262295,0.23750000000000002,-20.5280562633211,0.00025277231465770014,Watermarked +2106,0.6721311475409836,0.35,-8.241430969943103,0.00374033200083373,Watermarked +2107,0.6065573770491803,0.1958333333333333,-10.95264116575592,0.0016294169815328763,Watermarked +2108,0.36065573770491804,0.26666666666666666,-2.270928029445486,0.10783275809661891,Watermarked +2109,0.5081967213114754,0.2416666666666667,-11.07944631333403,0.0015751594215650242,Watermarked +2110,0.3770491803278688,0.25,-2.8812045893326337,0.063466587684043,Watermarked +2111,0.48333333333333334,0.19904371584699454,-12.288393956261835,0.0011607197809402983,Watermarked +2112,0.5333333333333333,0.23633879781420766,-4.584804480637148,0.019485678388664683,Watermarked +2113,0.4,0.24881602914389797,-3.563314918926205,0.07052714913781105,Watermarked +2114,0.39344262295081966,0.19583333333333333,-4.628326083672306,0.018994819352054024,Watermarked +2115,0.5,0.298155737704918,-4.033530534554061,0.027402529576258054,Watermarked +2116,0.48333333333333334,0.2693306010928962,-6.652844379359568,0.006921592104201834,Watermarked +2117,0.4,0.22404371584699456,-12.321363422263417,0.0011515711663196136,Watermarked +2118,0.5,0.2528688524590164,-9.302810429704943,0.002629366644977586,Watermarked +2119,0.55,0.2778688524590164,-4.536961611771408,0.020044437314205348,Watermarked +2120,0.5833333333333334,0.29453551912568304,-13.407344687092644,0.0008970413965513573,Watermarked +2121,0.8360655737704918,0.22083333333333333,-33.295454627318755,5.9553511839070765e-05,Watermarked +2122,0.6,0.257172131147541,-22.28671843401477,0.0001977851067158647,Watermarked +2123,0.6721311475409836,0.14166666666666666,-49.30752240433902,1.836912647385703e-05,Watermarked +2124,0.7868852459016393,0.19166666666666665,-18.866186551774142,0.00032511960811709917,Watermarked +2125,0.7049180327868853,0.17500000000000002,-13.259465772580212,0.0009269807118292367,Watermarked +2126,0.6333333333333333,0.2614071038251366,-11.56614172997057,0.00138784136278078,Watermarked +2127,0.4918032786885246,0.26666666666666666,-3.4878013959288654,0.039830015747165624,Watermarked +2128,0.48333333333333334,0.22418032786885247,-17.367164105589854,0.0004160299549552091,Watermarked +2129,0.36666666666666664,0.23654371584699457,-7.528860163609132,0.0048569805574603395,Watermarked +2130,0.5081967213114754,0.22916666666666666,-7.321262735084583,0.005263650485292702,Watermarked +2131,0.5737704918032787,0.25,-9.922022842122464,0.0021777725182429894,Watermarked +2132,0.6885245901639344,0.14166666666666666,-27.567153326380936,0.00010477109294231701,Watermarked +2133,0.7704918032786885,0.19999999999999998,-17.482855438271482,0.00040788898005848117,Watermarked +2134,0.43333333333333335,0.2941939890710382,-4.012426942160979,0.027781897561995554,Watermarked +2135,0.8524590163934426,0.275,-22.287637793898487,0.00019776074953457593,Watermarked +2136,0.4098360655737705,0.3,-3.441600870206092,0.04118787585017441,Watermarked +2137,0.48333333333333334,0.3110655737704918,-4.021031471649375,0.027626407165013953,Watermarked +2138,0.43333333333333335,0.28572404371584703,-3.0768255794880175,0.054270837524734016,Watermarked +2139,0.55,0.2030737704918033,-11.28407580595368,0.0014925480755074861,Watermarked +2140,0.3770491803278688,0.26666666666666666,-2.4183597074545036,0.09430740171744867,Watermarked +2141,0.36666666666666664,0.24453551912568308,-3.448799623718753,0.0409724672224204,Watermarked +2142,0.36666666666666664,0.21550546448087432,-2.656953690777749,0.07653752004862353,Watermarked +2143,0.639344262295082,0.25416666666666665,-36.732973581097305,4.437567678781254e-05,Watermarked +2144,0.4426229508196721,0.2583333333333333,-9.290054918133373,0.002639919880417069,Watermarked +2145,0.639344262295082,0.23333333333333334,-9.555035305421034,0.0024316941386992286,Watermarked +2146,0.45901639344262296,0.25,-5.347493377444,0.012790241471607905,Watermarked +2147,0.55,0.2739071038251366,-24.893769141544713,0.00014212863367693852,Watermarked +2148,0.35,0.26994535519125684,-3.13501995090005,0.051863239303626886,Watermarked +2149,0.7213114754098361,0.2875,-13.79033606511491,0.0008252499042687573,Watermarked +2150,0.5409836065573771,0.26666666666666666,-3.469865323288583,0.04035029930545199,Watermarked +2151,0.5,0.2864071038251366,-9.166199952585503,0.00274536805342564,Watermarked +2152,0.45,0.27363387978142073,-4.238851608241817,0.02403813124545163,Watermarked +2153,0.6721311475409836,0.23750000000000002,-12.680742877830454,0.001057786978105176,Watermarked +2154,0.5573770491803278,0.21250000000000002,-5.6891750976260695,0.010764907709515388,Watermarked +2155,0.5901639344262295,0.22916666666666669,-5.917936455036488,0.009638872740618382,Watermarked +2156,0.5409836065573771,0.27499999999999997,-7.078346628927314,0.005798537274283227,Watermarked +2157,0.5,0.2573087431693989,-12.130613891363168,0.0012058641450002766,Watermarked +2158,0.38333333333333336,0.25314207650273224,-3.7198098893146967,0.03381158141645187,Watermarked +2159,0.36666666666666664,0.29453551912568304,-4.000535767249803,0.02799863936714751,Watermarked +2160,0.45,0.1867486338797814,-18.377401251596062,0.00035156723441787536,Watermarked +2161,0.5737704918032787,0.2833333333333333,-12.870123433871873,0.0010124261840382058,Watermarked +2162,0.4426229508196721,0.2625,-4.652549903587426,0.018728552464152025,Watermarked +2163,0.4666666666666667,0.24863387978142074,-5.629498300891231,0.01108671624446389,Watermarked +2164,0.47540983606557374,0.22916666666666669,-4.253993084342693,0.023811667807213974,Watermarked +2165,0.4918032786885246,0.22916666666666666,-10.55444127076214,0.0018167912040410702,Watermarked +2166,0.45,0.28237704918032785,-9.511627906976749,0.0024642759907036603,Watermarked +2167,0.5573770491803278,0.29166666666666663,-55.22686591346667,1.307697160598242e-05,Watermarked +2168,0.5081967213114754,0.23333333333333334,-5.497267759562843,0.011845956731503078,Watermarked +2169,0.43333333333333335,0.2941256830601093,-2.1752046582440823,0.1178783573581168,Watermarked +2170,0.4098360655737705,0.325,-3.3934426229508206,0.04266659593484531,Watermarked +2171,0.5573770491803278,0.18333333333333335,-10.579553917424954,0.0018041539811479287,Watermarked +2172,0.4,0.20737704918032787,-4.581868223019867,0.019519384720467742,Watermarked +2173,0.5,0.31489071038251365,-3.002786519391192,0.05754103974524036,Watermarked +2174,0.31666666666666665,0.23674863387978146,-3.2609722394321854,0.04709695971232935,Watermarked +2175,0.48333333333333334,0.2693306010928962,-5.216676982893284,0.013697022421412816,Watermarked +2176,0.5833333333333334,0.30703551912568305,-26.318257342843076,0.00012035017922562646,Watermarked +2177,0.6,0.27363387978142073,-11.501316579943088,0.0014110204730729838,Watermarked +2178,0.4918032786885246,0.28750000000000003,-6.210497654675213,0.00841340269771356,Watermarked +2179,0.5901639344262295,0.20833333333333334,-23.92854336374633,0.00015995510518182317,Watermarked +2180,0.36065573770491804,0.3,-2.5734050069412073,0.08224938731599696,Watermarked +2181,0.47540983606557374,0.2833333333333333,-3.914708631165495,0.029629359927274702,Watermarked +2182,0.5081967213114754,0.3,-5.782581278907251,0.010285339228476416,Watermarked +2183,0.4666666666666667,0.2489754098360656,-9.227144083318263,0.002692797704338326,Watermarked +2184,0.5901639344262295,0.275,-4.488369320415114,0.02063340350086202,Watermarked +2185,0.5409836065573771,0.2791666666666667,-7.354405200045387,0.00519578998775744,Watermarked +2186,0.4098360655737705,0.2708333333333333,-9.766999910411144,0.002280530521151582,Watermarked +2187,0.3333333333333333,0.22659380692167577,-9.085817324099237,0.011897809590563803,Watermarked +2188,0.5573770491803278,0.25416666666666665,-6.766299230685365,0.0065959876919741,Watermarked +2189,0.7213114754098361,0.27499999999999997,-8.885189155139434,0.0030061660806281146,Watermarked +2190,0.48333333333333334,0.34446721311475414,-3.3256460363533833,0.04486462240497135,Watermarked +2191,0.65,0.26584699453551913,-15.779833617982495,0.0005532481835581896,Watermarked +2192,0.5666666666666667,0.2067622950819672,-6.043879310010205,0.009084400202355151,Watermarked +2193,0.6229508196721312,0.225,-9.957417838496047,0.0021551750735026,Watermarked +2194,0.4426229508196721,0.2583333333333333,-3.3466253328190545,0.04416940727951125,Watermarked +2195,0.45,0.2692622950819672,-4.2061085388157045,0.02453744768447861,Watermarked +2196,0.45,0.23654371584699452,-14.867978061029115,0.0006602177614744704,Watermarked +2197,0.6229508196721312,0.1875,-34.83606557377048,5.20111686902031e-05,Watermarked +2198,0.5833333333333334,0.20710382513661202,-10.283901901516982,0.001960684970337362,Watermarked +2199,0.4666666666666667,0.28592896174863386,-4.828799478606252,0.01693007546409189,Watermarked +2200,0.7213114754098361,0.3416666666666667,-7.851608784383327,0.004303261311903815,Watermarked +2201,0.6065573770491803,0.37916666666666665,-6.507337920439277,0.007370864736714179,Watermarked +2202,0.6666666666666666,0.489275956284153,-6.06668077104039,0.00898852350687363,Watermarked +2203,0.5833333333333334,0.373155737704918,-5.4116120970679615,0.01237436958129694,Watermarked +2204,0.4918032786885246,0.3875,-3.170655272045821,0.050455235210516786,Watermarked +2205,0.5901639344262295,0.4125,-5.932056727240316,0.0095745359301512,Watermarked +2206,0.6065573770491803,0.39999999999999997,-4.741062246648036,0.017795875960800792,Watermarked +2207,0.6557377049180327,0.39583333333333337,-6.819438116767888,0.006450408520510395,Watermarked +2208,0.6166666666666667,0.41427595628415304,-3.5454803247028934,0.03821378114440935,Watermarked +2209,0.5737704918032787,0.37083333333333335,-10.982657455064276,0.0016163522474045274,Watermarked +2210,0.5833333333333334,0.4810109289617486,-2.397936955599478,0.09605254927181417,Watermarked +2211,0.6557377049180327,0.39166666666666666,-4.380390416901878,0.022024716850259,Watermarked +2212,0.5245901639344263,0.36249999999999993,-5.412060981830556,0.012371520890864776,Watermarked +2213,0.5666666666666667,0.4185792349726776,-3.1318929933163084,0.05198914240331166,Watermarked +2214,0.6,0.3980191256830601,-6.2294612621620695,0.008341210430613992,Watermarked +2215,0.55,0.43142076502732246,-11.855239270966623,0.00129040506090979,Watermarked +2216,0.5166666666666667,0.4312158469945355,-2.769971026007625,0.06957017387261798,Watermarked +2217,0.5666666666666667,0.3818306010928962,-4.847117449604144,0.016756266049925205,Watermarked +2218,0.4666666666666667,0.4064890710382514,-1.6875505812729021,0.19008194720260654,Watermarked +2219,0.5573770491803278,0.4,-8.742170122442447,0.003151555065586373,Watermarked +2220,0.6,0.47274590163934427,-3.8037548196566555,0.03192304772258143,Watermarked +2221,0.6666666666666666,0.3689207650273224,-6.200033744114341,0.00845358911672797,Watermarked +2222,0.6229508196721312,0.425,-12.405173285892113,0.0011287421498101565,Watermarked +2223,0.5833333333333334,0.3605874316939891,-5.648887116271631,0.010980796736842062,Watermarked +2224,0.5833333333333334,0.39364754098360655,-4.293991625472901,0.023226546554816672,Watermarked +2225,0.5833333333333334,0.40614754098360656,-4.422690436146169,0.02146562039496988,Watermarked +2226,0.6166666666666667,0.38148907103825136,-8.30926126344939,0.0036525000991545153,Watermarked +2227,0.6065573770491803,0.38749999999999996,-3.4641024571780417,0.04051930159431608,Watermarked +2228,0.6166666666666667,0.3691939890710383,-6.170366319105602,0.008568905214827621,Watermarked +2229,0.5573770491803278,0.4041666666666667,-3.05362313626129,0.05526992091860083,Watermarked +2230,0.6833333333333333,0.42711748633879776,-7.960605911691153,0.004135187577765976,Watermarked +2231,0.6666666666666666,0.45163934426229513,-3.2153935344715583,0.04875516994578165,Watermarked +2232,0.55,0.43995901639344265,-3.5143217545081513,0.03907627630046548,Watermarked +2233,0.6666666666666666,0.3816256830601093,-11.100144808896685,0.0015665315397856033,Watermarked +2234,0.5333333333333333,0.4523224043715847,-2.5642179367732165,0.082909074165789,Watermarked +2235,0.55,0.4187158469945355,-3.611089663775107,0.03647560832552664,Watermarked +2236,0.5666666666666667,0.4362021857923497,-2.402996731954332,0.09561654218317737,Watermarked +2237,0.75,0.44371584699453553,-9.529761387107365,0.0024505949878125062,Watermarked +2238,0.4666666666666667,0.42315573770491804,-2.2737870570417353,0.10754893601572461,Watermarked +2239,0.6557377049180327,0.4083333333333333,-7.346004713131698,0.005212880633794737,Watermarked +2240,0.5737704918032787,0.35833333333333334,-4.3698631465799025,0.022166781498566,Watermarked +2241,0.6885245901639344,0.4,-16.027311891144493,0.0005282450921087477,Watermarked +2242,0.5,0.40642076502732244,-3.4412842582299485,0.04119738288603227,Watermarked +2243,0.5409836065573771,0.36666666666666664,-3.8190928053365694,0.031592715134547304,Watermarked +2244,0.5573770491803278,0.4458333333333333,-2.433681073025334,0.09302352271601963,Watermarked +2245,0.65,0.3774590163934426,-8.635338782373333,0.00326628135311723,Watermarked +2246,0.55,0.41448087431693986,-2.7772482375001317,0.06914909220734049,Watermarked +2247,0.5573770491803278,0.3916666666666667,-2.4539091507908326,0.09136101012842844,Watermarked +2248,0.6229508196721312,0.425,-3.8366329971259767,0.031220308656298136,Watermarked +2249,0.5245901639344263,0.425,-2.4919181614361894,0.08833429775357093,Watermarked +2250,0.5833333333333334,0.4394808743169399,-3.2347368286888813,0.048042559031082566,Watermarked +2251,0.5,0.41051912568306015,-3.2435150269952304,0.04772351523132277,Watermarked +2252,0.5737704918032787,0.4291666666666667,-3.4762904569519018,0.04016293189095533,Watermarked +2253,0.5166666666666667,0.4148224043715847,-5.582808024011134,0.011347326397319939,Watermarked +2254,0.6,0.39385245901639343,-4.622249873571463,0.019062378557272608,Watermarked +2255,0.55,0.42288251366120216,-2.2047766760939767,0.11465918253571977,Watermarked +2256,0.48333333333333334,0.39392076502732243,-3.234755633152603,0.048041872703208165,Watermarked +2257,0.5333333333333333,0.4519808743169399,-1.7442611130916705,0.17946189402091586,Watermarked +2258,0.6065573770491803,0.3791666666666667,-4.961251862891206,0.01572399871994463,Watermarked +2259,0.7213114754098361,0.3041666666666667,-10.94515337164297,0.0016326978989827164,Watermarked +2260,0.48333333333333334,0.4235655737704918,-2.1896107986499023,0.11629675807965059,Watermarked +2261,0.5333333333333333,0.43948087431693983,-2.3754283562972716,0.09802174273100718,Watermarked +2262,0.6721311475409836,0.4208333333333333,-4.533288093479356,0.020088194824742056,Watermarked +2263,0.5833333333333334,0.3310109289617486,-3.076310185278194,0.05429278214985614,Watermarked +2264,0.55,0.43586065573770494,-3.3338420681873173,0.04459138016214428,Watermarked +2265,0.5333333333333333,0.4228825136612022,-3.399472771732523,0.042477781423474185,Watermarked +2266,0.5901639344262295,0.4666666666666667,-2.9838938061318583,0.0584148664790183,Watermarked +2267,0.639344262295082,0.3833333333333333,-8.868478725093105,0.003022689256232251,Watermarked +2268,0.6666666666666666,0.4437841530054645,-11.678549121160296,0.0013488323998950805,Watermarked +2269,0.5166666666666667,0.4271174863387978,-3.7411614919740814,0.03331794416540563,Watermarked +2270,0.5666666666666667,0.4189207650273224,-4.8323202058438195,0.016896487730970963,Watermarked +2271,0.5666666666666667,0.344672131147541,-3.571560837934814,0.03751043917785419,Watermarked +2272,0.5833333333333334,0.3942622950819672,-7.793998738706418,0.004395749962862252,Watermarked +2273,0.5833333333333334,0.47677595628415304,-2.363289165755069,0.09910414730873617,Watermarked +2274,0.55,0.43586065573770494,-4.152645529463587,0.025381891014694326,Watermarked +2275,0.5166666666666667,0.40225409836065573,-4.3808607027131465,0.022018397914196755,Watermarked +2276,0.639344262295082,0.41250000000000003,-7.574148554850093,0.0047738189211758006,Watermarked +2277,0.5737704918032787,0.45833333333333337,-2.975985583098297,0.058785578261159106,Watermarked +2278,0.5166666666666667,0.4522540983606557,-3.550343434787011,0.03808136417851271,Watermarked +2279,0.5409836065573771,0.4,-3.9876185693143014,0.028236554956465203,Watermarked +2280,0.6557377049180327,0.3416666666666667,-7.253159939163658,0.005406805253936257,Watermarked +2281,0.6229508196721312,0.38749999999999996,-3.8840535871493516,0.030241314676875894,Watermarked +2282,0.7666666666666667,0.36065573770491804,-12.408708085543239,0.001127792567025593,Watermarked +2283,0.5573770491803278,0.35000000000000003,-6.992150468759068,0.006005502007535316,Watermarked +2284,0.5166666666666667,0.4148907103825137,-9.202675289866411,0.0027137429423814024,Watermarked +2285,0.6666666666666666,0.42288251366120216,-4.288438027587506,0.023306668223673156,Watermarked +2286,0.45901639344262296,0.3666666666666667,-2.327678697056396,0.10236424529812158,Watermarked +2287,0.7213114754098361,0.38333333333333336,-8.1660987570467,0.0038411801830783828,Watermarked +2288,0.6166666666666667,0.43196721311475417,-3.728410507789727,0.03361162082090968,Watermarked +2289,0.5409836065573771,0.38333333333333336,-6.688534635157998,0.0068169278980430605,Watermarked +2290,0.5166666666666667,0.39385245901639343,-3.766631201193383,0.03274110546757008,Watermarked +2291,0.5333333333333333,0.3813524590163934,-3.0906245783493898,0.05368742829459938,Watermarked +2292,0.5737704918032787,0.4,-4.7424625152411375,0.01778161142649878,Watermarked +2293,0.6,0.38162568306010936,-6.039077465113362,0.009104762087317147,Watermarked +2294,0.6229508196721312,0.5041666666666667,-3.021862808739443,0.05667530663209718,Watermarked +2295,0.55,0.4146857923497268,-3.8865083819708626,0.030191715950954285,Watermarked +2296,0.55,0.476912568306011,-3.077135925982908,0.0542576288739498,Watermarked +2297,0.6885245901639344,0.39583333333333337,-15.840015885207144,0.0005470254169839911,Watermarked +2298,0.7540983606557377,0.39999999999999997,-11.93916601780205,0.0012638276378236352,Watermarked +2299,0.5166666666666667,0.40218579234972673,-2.5598239902355586,0.08322688778357085,Watermarked +2300,0.7213114754098361,0.36666666666666664,-11.112432660516188,0.0015614391657328315,Watermarked +2301,0.6065573770491803,0.45416666666666666,-2.293338139417192,0.10563206329335763,Watermarked +2302,0.47540983606557374,0.37083333333333335,-2.937540923889767,0.06063034832622183,Watermarked +2303,0.6065573770491803,0.4,-7.838301347827194,0.004324395459725133,Watermarked +2304,0.6,0.39344262295081966,-2.704003786107557,0.0735363944189725,Watermarked +2305,0.5245901639344263,0.4291666666666667,-2.2939904432507365,0.10556882390727751,Watermarked +2306,0.639344262295082,0.43333333333333335,-5.721871764168063,0.010593767203391514,Watermarked +2307,0.6557377049180327,0.4375,-9.510009673164706,0.002465501797241344,Watermarked +2308,0.5333333333333333,0.4521174863387978,-3.296192817472393,0.04586424145751324,Watermarked +2309,0.5333333333333333,0.373155737704918,-3.7418090166847584,0.03330311842719912,Watermarked +2310,0.5666666666666667,0.39002732240437155,-5.154995937878714,0.014153740439881685,Watermarked +2311,0.5666666666666667,0.39767759562841526,-2.5958139567349727,0.08066720661779468,Watermarked +2312,0.5666666666666667,0.4650273224043716,-3.1399952826589113,0.051663705233341434,Watermarked +2313,0.6166666666666667,0.3982923497267759,-4.422992997264673,0.021461687900979676,Watermarked +2314,0.6166666666666667,0.38934426229508196,-3.7175780367875846,0.03386371975978312,Watermarked +2315,0.48333333333333334,0.4191256830601093,-2.474712698219785,0.08968894024591773,Watermarked +2316,0.6885245901639344,0.4208333333333334,-6.523182985505984,0.007320113516282464,Watermarked +2317,0.639344262295082,0.4375,-5.669780163353537,0.010868137779207396,Watermarked +2318,0.6,0.34795081967213115,-4.836426389706701,0.01685742415885029,Watermarked +2319,0.47540983606557374,0.27499999999999997,-6.209471659985009,0.008417331899586392,Watermarked +2320,0.6666666666666666,0.4395491803278688,-6.062960519560702,0.009004075645090898,Watermarked +2321,0.5166666666666667,0.38633879781420766,-3.142740772481161,0.051554012753716585,Watermarked +2322,0.5166666666666667,0.3895491803278689,-2.647776278064909,0.07714046268331175,Watermarked +2323,0.5737704918032787,0.4375,-3.5755004055064217,0.037405636887797994,Watermarked +2324,0.5666666666666667,0.47288251366120215,-1.789503264890946,0.17147429025836847,Watermarked +2325,0.5833333333333334,0.3976092896174863,-2.616716369900444,0.07922508793306127,Watermarked +2326,0.6885245901639344,0.3458333333333334,-12.446270789290725,0.0011177670196131112,Watermarked +2327,0.5666666666666667,0.4144125683060109,-3.3818058014152355,0.04303398980095592,Watermarked +2328,0.4666666666666667,0.40225409836065573,-1.9432287224055769,0.14724565777265242,Watermarked +2329,0.639344262295082,0.3625,-6.568076588878649,0.007178794023354771,Watermarked +2330,0.5166666666666667,0.38989071038251366,-3.8433173783046315,0.031079871545519045,Watermarked +2331,0.5666666666666667,0.41058743169398904,-5.412428171026809,0.012369191290195588,Watermarked +2332,0.6065573770491803,0.4291666666666667,-4.446685303445383,0.021156625841990514,Watermarked +2333,0.6065573770491803,0.41666666666666663,-4.468875866263539,0.020875975290483854,Watermarked +2334,0.5166666666666667,0.4230191256830601,-2.1008219346441472,0.12646621272270542,Watermarked +2335,0.5833333333333334,0.40710382513661203,-4.176894082046699,0.02499431409045767,Watermarked +2336,0.65,0.41065573770491803,-6.713645725875071,0.0067445330979899526,Watermarked +2337,0.5573770491803278,0.425,-2.8842564548659144,0.06330874589287337,Watermarked +2338,0.6229508196721312,0.45416666666666666,-6.130104668195616,0.008728726955771435,Watermarked +2339,0.7049180327868853,0.37083333333333335,-8.031429676406121,0.004030601648600872,Watermarked +2340,0.6,0.4480191256830601,-3.7683602379617436,0.032702411681861464,Watermarked +2341,0.48333333333333334,0.41885245901639345,-1.6093008135111428,0.20592487686100341,Watermarked +2342,0.7049180327868853,0.37083333333333335,-6.267404269385084,0.008199190432380252,Watermarked +2343,0.55,0.4438524590163934,-5.090988202057138,0.014648921763255886,Watermarked +2344,0.5081967213114754,0.3083333333333333,-18.57762175574247,0.0003403993871209538,Watermarked +2345,0.45,0.40232240437158473,-2.44235164169727,0.09230641363259132,Watermarked +2346,0.48333333333333334,0.39801912568306014,-2.3401785284145156,0.10120527554802543,Watermarked +2347,0.5,0.3650273224043715,-3.293760329942494,0.0459480546557983,Watermarked +2348,0.4666666666666667,0.392167577413479,-1.8864910288697916,0.19986672981680156,Watermarked +2349,0.6065573770491803,0.45833333333333337,-4.811368296463101,0.017097654477865952,Watermarked +2350,0.5833333333333334,0.3773224043715847,-9.313966981164503,0.002620182057561568,Watermarked +2351,0.5666666666666667,0.4148224043715847,-4.737661283448938,0.017830583546120875,Watermarked +2352,0.639344262295082,0.4583333333333333,-4.666486269355321,0.018577558529827218,Watermarked +2353,0.6666666666666666,0.43920765027322406,-4.46624735000153,0.02090896608594079,Watermarked +2354,0.5166666666666667,0.4187158469945355,-1.9672355178709884,0.1438337407120422,Watermarked +2355,0.6065573770491803,0.4,-8.419686273573582,0.0035152556902358035,Watermarked +2356,0.5245901639344263,0.44166666666666665,-3.447065049817408,0.041024239621887254,Watermarked +2357,0.5573770491803278,0.45,-3.945284708663067,0.029034743151997156,Watermarked +2358,0.5333333333333333,0.41045081967213115,-3.0577746633809832,0.055089469506326795,Watermarked +2359,0.65,0.4066256830601093,-14.185315533396013,0.0007589934893947729,Watermarked +2360,0.5833333333333334,0.3400273224043716,-5.098953338267703,0.0145860830962082,Watermarked +2361,0.6166666666666667,0.4058743169398907,-3.305417706655741,0.04554814619701029,Watermarked +2362,0.5573770491803278,0.4125,-2.2397642698297733,0.11098584190703364,Watermarked +2363,0.6229508196721312,0.4041666666666667,-6.036355410889254,0.009116331366960937,Watermarked +2364,0.4918032786885246,0.4125,-3.074072401423919,0.05438819286539707,Watermarked +2365,0.6833333333333333,0.3903005464480874,-10.873299213730528,0.0016646323342411745,Watermarked +2366,0.6885245901639344,0.36249999999999993,-8.995168397182145,0.0029003471322019392,Watermarked +2367,0.5737704918032787,0.4208333333333333,-6.6644480933382715,0.006887332838366486,Watermarked +2368,0.5409836065573771,0.425,-2.678526658699412,0.07514300545940161,Watermarked +2369,0.5666666666666667,0.43558743169398906,-8.104167446872971,0.003926796249409396,Watermarked +2370,0.5833333333333334,0.4191256830601093,-4.775358960423008,0.017450729445301273,Watermarked +2371,0.7049180327868853,0.40416666666666673,-6.290438634313473,0.008114520022772869,Watermarked +2372,0.7333333333333333,0.3650956284153005,-6.627801681197047,0.0069962968928984955,Watermarked +2373,0.5333333333333333,0.42295081967213116,-2.3568974575090693,0.09967991090436344,Watermarked +2374,0.5333333333333333,0.41045081967213115,-4.3952411570175425,0.021826304453140925,Watermarked +2375,0.639344262295082,0.41250000000000003,-3.4877149221186383,0.03983250358959142,Watermarked +2376,0.6,0.4026639344262295,-4.885900809261954,0.016395862833158793,Watermarked +2377,0.7049180327868853,0.37916666666666665,-7.6296187670353355,0.004674501764841112,Watermarked +2378,0.6229508196721312,0.4458333333333333,-6.638664993068123,0.006963760953222826,Watermarked +2379,0.7166666666666667,0.41912568306010933,-17.09897132088851,0.00043575164889100897,Watermarked +2380,0.6833333333333333,0.44829234972677595,-5.766710653623844,0.010364820031007032,Watermarked +2381,0.639344262295082,0.4125,-5.5278108432930315,0.01166468471503337,Watermarked +2382,0.5901639344262295,0.425,-10.350486199201907,0.0019239067953852225,Watermarked +2383,0.5166666666666667,0.4314890710382514,-3.272785517027942,0.046678870059437404,Watermarked +2384,0.6065573770491803,0.39583333333333337,-10.70159618814971,0.0017443718008383813,Watermarked +2385,0.6557377049180327,0.4,-4.125557008441323,0.025824079480551702,Watermarked +2386,0.6666666666666666,0.39385245901639343,-8.973702530064598,0.0029206091675186585,Watermarked +2387,0.6666666666666666,0.41885245901639345,-12.364286659593969,0.0011398034481367935,Watermarked +2388,0.4426229508196721,0.4,-2.801466195764125,0.06777041139462144,Watermarked +2389,0.6721311475409836,0.4166666666666667,-7.828769077961604,0.004339618418212919,Watermarked +2390,0.6721311475409836,0.3458333333333333,-12.230197713801166,0.0011771054889125297,Watermarked +2391,0.5081967213114754,0.4041666666666667,-4.3462339347627506,0.02249000005004558,Watermarked +2392,0.6885245901639344,0.43333333333333335,-9.683827326548414,0.0023383191790880323,Watermarked +2393,0.5,0.4439207650273224,-2.1366037206586563,0.12224492481984166,Watermarked +2394,0.7704918032786885,0.41666666666666663,-23.25576104817017,0.00017417898616924437,Watermarked +2395,0.6,0.3983606557377049,-14.711044641258004,0.0006813380311103391,Watermarked +2396,0.5901639344262295,0.44166666666666665,-3.8282796790862137,0.0313969565110335,Watermarked +2397,0.55,0.3980874316939891,-5.915897985549872,0.009648207488286976,Watermarked +2398,0.6557377049180327,0.4291666666666667,-4.396123870194077,0.021814583998735686,Watermarked +2399,0.7704918032786885,0.375,-27.40047589022894,0.00010668857406321375,Watermarked +2400,0.6721311475409836,0.4416666666666667,-5.458841849494081,0.012079282996343666,Watermarked +2401,0.5737704918032787,0.39166666666666666,-7.569913857342007,0.004781514945095522,Watermarked +2402,0.5666666666666667,0.44781420765027324,-3.9576539242406734,0.02879855831206138,Watermarked +2403,0.5737704918032787,0.43333333333333335,-3.3050391298697224,0.04556106387896335,Watermarked +2404,0.6229508196721312,0.275,-20.058020052863462,0.0002708532723360041,Watermarked +2405,0.7,0.38196721311475407,-10.445379856844973,0.001873056150700024,Watermarked +2406,0.5833333333333334,0.35655737704918034,-5.1034502548832,0.01455076115150943,Watermarked +2407,0.8032786885245902,0.3791666666666667,-15.40340748070682,0.0005943876559434028,Watermarked +2408,0.5245901639344263,0.375,-4.047798691785726,0.02714981201550245,Watermarked +2409,0.5,0.44398907103825136,-3.5630178449635013,0.0377389945733,Watermarked +2410,0.7049180327868853,0.4666666666666667,-6.095443431544011,0.008869465900928337,Watermarked +2411,0.55,0.4312841530054645,-3.2330608226780675,0.0481037799750477,Watermarked +2412,0.5409836065573771,0.4416666666666667,-2.8905475311707214,0.06298491773711792,Watermarked +2413,0.55,0.4110655737704918,-5.966922212523975,0.009418071176895705,Watermarked +2414,0.75,0.36495901639344264,-15.214865853342191,0.0006165289089404656,Watermarked +2415,0.6721311475409836,0.3916666666666667,-33.655737704917975,5.766533760280624e-05,Watermarked +2416,0.5666666666666667,0.4230874316939891,-3.063975004864087,0.054821341360087615,Watermarked +2417,0.5409836065573771,0.42500000000000004,-7.268455569166599,0.005374210400514755,Watermarked +2418,0.6065573770491803,0.37083333333333335,-7.4933789577565,0.004923473774846551,Watermarked +2419,0.5737704918032787,0.48750000000000004,-4.140983606557374,0.025571051486798255,Watermarked +2420,0.6229508196721312,0.45416666666666666,-5.024423429015315,0.015188120980305192,Watermarked +2421,0.6666666666666666,0.34426229508196726,-10.200056186904932,0.0020083355064274685,Watermarked +2422,0.75,0.4724043715846995,-4.432281354966694,0.021341416432768157,Watermarked +2423,0.6,0.435724043715847,-7.812794009945692,0.004365289535356393,Watermarked +2424,0.5245901639344263,0.37083333333333335,-5.133817644374452,0.014315130666193885,Watermarked +2425,0.5573770491803278,0.42083333333333334,-5.4872114695067635,0.011906446056017608,Watermarked +2426,0.6885245901639344,0.44583333333333336,-5.613383822262529,0.011175768196393236,Watermarked +2427,0.6721311475409836,0.4666666666666667,-3.3346995013274205,0.044562916865554185,Watermarked +2428,0.6,0.3775273224043716,-6.826452300266374,0.006431509604653321,Watermarked +2429,0.6166666666666667,0.3818989071038251,-7.577650824830856,0.004767466337140692,Watermarked +2430,0.5666666666666667,0.4771857923497268,-2.244146678485596,0.11053578481406542,Watermarked +2431,0.6065573770491803,0.35833333333333334,-3.3233585459170785,0.044941262424214,Watermarked +2432,0.5245901639344263,0.4125000000000001,-6.066153107963813,0.008990727202934024,Watermarked +2433,0.6885245901639344,0.37916666666666665,-29.50232973023399,8.552802430562091e-05,Watermarked +2434,0.65,0.35628415300546445,-6.079870046364211,0.008933670268337511,Watermarked +2435,0.7333333333333333,0.42356557377049187,-9.685668717829044,0.002337018895349377,Watermarked +2436,0.639344262295082,0.38333333333333336,-11.898312771791307,0.001276673082635795,Watermarked +2437,0.65,0.45642076502732243,-6.157320526109763,0.008620267599043325,Watermarked +2438,0.6,0.35273224043715845,-11.787092032898832,0.0013125340009585288,Watermarked +2439,0.6557377049180327,0.48750000000000004,-4.044451296869408,0.027208830672879287,Watermarked +2440,0.6065573770491803,0.32916666666666666,-8.618610742527247,0.003284743377152693,Watermarked +2441,0.8360655737704918,0.3625,-21.607919158456742,0.00021691691845571816,Watermarked +2442,0.6065573770491803,0.425,-12.578664061524803,0.0010833606619678423,Watermarked +2443,0.5737704918032787,0.4166666666666667,-5.772363123361995,0.010336420243029285,Watermarked +2444,0.5166666666666667,0.43968579234972677,-2.8833841562530913,0.06335381086618001,Watermarked +2445,0.5833333333333334,0.4685109289617486,-3.74670122389729,0.03319137901399563,Watermarked +2446,0.6065573770491803,0.3916666666666667,-4.201654069631227,0.02460640768982449,Watermarked +2447,0.5333333333333333,0.4519808743169399,-3.088509743881098,0.05377632746313144,Watermarked +2448,0.6166666666666667,0.464412568306011,-3.9929846359991004,0.0281374053103564,Watermarked +2449,0.8688524590163934,0.4041666666666667,-111.52459016393443,1.5893997832869387e-06,Watermarked +2450,0.6557377049180327,0.3625,-10.053864168618263,0.002095156783314077,Watermarked +2451,0.639344262295082,0.46249999999999997,-4.30939544513269,0.023006171549745307,Watermarked +2452,0.6333333333333333,0.41878415300546445,-3.907188916118307,0.029777989336963344,Watermarked +2453,0.5333333333333333,0.4311475409836066,-2.3185802321951905,0.10321797628133385,Watermarked +2454,0.6,0.4395491803278689,-3.7611226340766057,0.03286477251791561,Watermarked +2455,0.5573770491803278,0.42500000000000004,-7.6310252969080015,0.004672019040052037,Watermarked +2456,0.48333333333333334,0.3772540983606557,-2.892862105157335,0.06286629800985993,Watermarked +2457,0.55,0.3484972677595628,-4.079918215864471,0.026591801968465816,Watermarked +2458,0.5666666666666667,0.35689890710382516,-3.012713968246357,0.057088449146294205,Watermarked +2459,0.5833333333333334,0.4519808743169399,-3.585650713444847,0.03713733022490143,Watermarked +2460,0.6,0.39829234972677596,-6.228521035735816,0.008344770553796356,Watermarked +2461,0.5833333333333334,0.36495901639344264,-11.467033890261218,0.001423486681886134,Watermarked +2462,0.6885245901639344,0.37083333333333335,-18.492347410557112,0.00034509721387531697,Watermarked +2463,0.639344262295082,0.35833333333333334,-7.031379503531193,0.005910122809393594,Watermarked +2464,0.6666666666666666,0.4273224043715847,-7.050049697276166,0.005865429795204114,Watermarked +2465,0.5333333333333333,0.36475409836065575,-3.2441722541392934,0.047699736310779436,Watermarked +2466,0.6229508196721312,0.40416666666666673,-4.882283495355984,0.016429048371143395,Watermarked +2467,0.48333333333333334,0.40286885245901644,-2.273916596472962,0.10753609772746466,Watermarked +2468,0.7166666666666667,0.43176229508196723,-4.998106522840042,0.01540842738464649,Watermarked +2469,0.6833333333333333,0.44036885245901636,-2.6121434779788526,0.07953784934392538,Watermarked +2470,0.5666666666666667,0.48524590163934433,-1.929654958496209,0.1492167828881105,Watermarked +2471,0.5333333333333333,0.4275273224043716,-3.5561117736258945,0.037925056755999366,Watermarked +2472,0.639344262295082,0.4625,-4.040627784984222,0.027276446171401562,Watermarked +2473,0.639344262295082,0.475,-6.831662693460199,0.006417517963111248,Watermarked +2474,0.7049180327868853,0.35833333333333334,-24.012092343181642,0.0001582981185228392,Watermarked +2475,0.7704918032786885,0.4041666666666667,-8.579921461021282,0.003327975019105188,Watermarked +2476,0.5666666666666667,0.3778688524590164,-5.101866069096539,0.014563191721761432,Watermarked +2477,0.7166666666666667,0.385724043715847,-7.664492373372008,0.004613454840878532,Watermarked +2478,0.48333333333333334,0.4031876138433515,-3.0047783082409976,0.09520452944787951,Watermarked +2479,0.6065573770491803,0.4375,-5.252662296910844,0.013439429896053543,Watermarked +2480,0.43333333333333335,0.35280054644808745,-2.803386864259781,0.06766254223000473,Watermarked +2481,0.7868852459016393,0.38333333333333336,-10.324506821548738,0.0019381469069513462,Watermarked +2482,0.6166666666666667,0.42288251366120216,-3.622802747830676,0.036175989328214676,Watermarked +2483,0.65,0.3319672131147541,-17.615818436997262,0.0003987923532231844,Watermarked +2484,0.6833333333333333,0.4146174863387978,-6.525168128577647,0.007313787597179912,Watermarked +2485,0.6,0.3689207650273224,-4.354954591021095,0.022370006686486543,Watermarked +2486,0.5737704918032787,0.3625,-4.5595686353563485,0.01977785439608666,Watermarked +2487,0.65,0.41475409836065574,-8.217472162729653,0.00377202241458133,Watermarked +2488,0.6333333333333333,0.4316256830601093,-6.118804569189945,0.008774285877481431,Watermarked +2489,0.5573770491803278,0.3958333333333333,-5.135277089276766,0.014303932115525365,Watermarked +2490,0.5666666666666667,0.4191256830601093,-3.801261952080425,0.03197715417981016,Watermarked +2491,0.6065573770491803,0.3666666666666667,-6.232543479694566,0.008329553737779013,Watermarked +2492,0.5901639344262295,0.4083333333333333,-8.2470608782833,0.0037329364018606546,Watermarked +2493,0.5666666666666667,0.41892076502732245,-5.9784922810858205,0.009366890251308508,Watermarked +2494,0.5666666666666667,0.4479508196721312,-2.7046377161768436,0.0734969637824313,Watermarked +2495,0.6833333333333333,0.38169398907103824,-8.926429305995807,0.002965896425199081,Watermarked +2496,0.6557377049180327,0.4208333333333333,-6.853539958645125,0.00635920568434066,Watermarked +2497,0.5833333333333334,0.4351775956284153,-3.62042968770528,0.03623643715847006,Watermarked +2498,0.7704918032786885,0.3666666666666667,-13.615808159190916,0.0008569802174432299,Watermarked +2499,0.6229508196721312,0.3833333333333333,-13.310561106566025,0.0009164879399678906,Watermarked +2500,0.7049180327868853,0.26666666666666666,-13.732158696046199,0.0008356507564616465,Watermarked +2501,0.5833333333333334,0.3980874316939891,-8.506737943489776,0.003411830999011348,Watermarked +2502,0.6885245901639344,0.44166666666666665,-5.438685446957326,0.01220408026603391,Watermarked +2503,0.48333333333333334,0.4064890710382514,-2.559704324213269,0.0832355641629743,Watermarked +2504,0.7,0.4148224043715847,-5.693637680357018,0.010741337381016519,Watermarked +2505,0.6166666666666667,0.4025273224043716,-8.890442589650126,0.0030009962178549105,Watermarked +2506,0.6557377049180327,0.42500000000000004,-6.851133053826303,0.006365586871051235,Watermarked +2507,0.6229508196721312,0.4791666666666667,-5.22212477804604,0.013657614104771973,Watermarked +2508,0.6166666666666667,0.3894808743169399,-4.915294791461667,0.01612940842830176,Watermarked +2509,0.8688524590163934,0.35,-25.41847552756969,0.0001335388647953177,Watermarked +2510,0.5666666666666667,0.37315573770491806,-3.2713284175710178,0.04673018421334356,Watermarked +2511,0.6333333333333333,0.43606557377049177,-5.107098757651382,0.014522185166915114,Watermarked +2512,0.6333333333333333,0.37315573770491806,-8.428620891983224,0.003504451310362902,Watermarked +2513,0.6,0.35273224043715845,-29.617790180570413,8.45343765080844e-05,Watermarked +2514,0.6557377049180327,0.44999999999999996,-4.3195919820040585,0.02286178010818133,Watermarked +2515,0.7377049180327869,0.3291666666666667,-15.836371260048626,0.000547399610373209,Watermarked +2516,0.5573770491803278,0.4333333333333333,-2.604375511062208,0.08007263680217473,Watermarked +2517,0.5409836065573771,0.42500000000000004,-4.639344262295081,0.01887310292166381,Watermarked +2518,0.6229508196721312,0.3958333333333333,-4.09708695591706,0.026299586172448756,Watermarked +2519,0.5409836065573771,0.3958333333333333,-3.537066558867117,0.03844426851884796,Watermarked +2520,0.6166666666666667,0.4230874316939891,-6.814259023242736,0.006464409944966311,Watermarked +2521,0.6721311475409836,0.3416666666666666,-30.71720234254342,7.580035597644182e-05,Watermarked +2522,0.7213114754098361,0.42916666666666664,-9.51209634369107,0.002463921302178941,Watermarked +2523,0.5833333333333334,0.39772313296903467,-11.857859091548148,0.00703694821787774,Watermarked +2524,0.7166666666666667,0.36461748633879776,-8.164949484411201,0.003842746397948636,Watermarked +2525,0.5409836065573771,0.5,-2.0077784776911325,0.1382803772888902,Watermarked +2526,0.6666666666666666,0.4025273224043716,-24.399378398771418,0.0001509087662519508,Watermarked +2527,0.6,0.40211748633879785,-3.4850713293771878,0.03990865507980857,Watermarked +2528,0.5409836065573771,0.4541666666666667,-2.176256072652707,0.11776205733944392,Watermarked +2529,0.5409836065573771,0.39166666666666666,-4.846843823072969,0.01675884522410491,Watermarked +2530,0.6333333333333333,0.4689890710382514,-2.3029347384916004,0.10470629811332137,Watermarked +2531,0.7377049180327869,0.3125,-40.55023240686837,3.3002033131333967e-05,Watermarked +2532,0.65,0.44330601092896177,-3.289205567147825,0.04610551507892302,Watermarked +2533,0.5333333333333333,0.3819672131147541,-2.224352555510678,0.11258619639823252,Watermarked +2534,0.5245901639344263,0.38333333333333336,-2.615568260013164,0.07930346961482511,Watermarked +2535,0.7666666666666667,0.3896857923497268,-13.06989618982596,0.0009673345468333082,Watermarked +2536,0.6229508196721312,0.44999999999999996,-2.9350727376464576,0.06075124544604779,Watermarked +2537,0.6557377049180327,0.38333333333333336,-4.592341249903026,0.0193995061866707,Watermarked +2538,0.6,0.3729508196721311,-5.094565492831203,0.014620656170024945,Watermarked +2539,0.7666666666666667,0.34446721311475414,-16.788036508322786,0.00046020497513463827,Watermarked +2540,0.5901639344262295,0.33333333333333337,-5.392319340084748,0.012497618930811919,Watermarked +2541,0.6065573770491803,0.41666666666666663,-12.480891064461987,0.0011086313738353398,Watermarked +2542,0.7213114754098361,0.45,-15.07113703400236,0.0006341498821599429,Watermarked +2543,0.7,0.38592896174863384,-7.972344974800964,0.004117606574879096,Watermarked +2544,0.5409836065573771,0.3791666666666667,-7.383406964717152,0.00513735130962263,Watermarked +2545,0.5081967213114754,0.3666666666666667,-3.4195951141068646,0.041855350747093845,Watermarked +2546,0.5901639344262295,0.48750000000000004,-2.6937228618928075,0.07417954696153008,Watermarked +2547,0.5245901639344263,0.3375,-8.980327868852457,0.0029143354515137163,Watermarked +2548,0.6333333333333333,0.3610655737704918,-6.352580462402448,0.007891754845695532,Watermarked +2549,0.45,0.3734972677595629,-4.334361862984616,0.022654697680077447,Watermarked +2550,0.6166666666666667,0.4599726775956284,-2.514156166273152,0.08662037793756441,Watermarked +2551,0.5166666666666667,0.360724043715847,-3.4306011747993215,0.041519810852172816,Watermarked +2552,0.6,0.44405737704918036,-3.2820666584674085,0.04635368887671906,Watermarked +2553,0.6666666666666666,0.4273224043715847,-6.912582855578067,0.006205274596049662,Watermarked +2554,0.5333333333333333,0.39849726775956285,-7.109187347867667,0.005726771511083016,Watermarked +2555,0.6885245901639344,0.3875,-7.23666133386256,0.005442256485075891,Watermarked +2556,0.5833333333333334,0.41523224043715845,-2.490304414512709,0.08846028307415578,Watermarked +2557,0.6833333333333333,0.4265710382513661,-3.6163076187603744,0.03634174375169587,Watermarked +2558,0.6885245901639344,0.35416666666666663,-13.436668405820718,0.000891257804744702,Watermarked +2559,0.55,0.3236338797814208,-3.600554934819319,0.03674779775320059,Watermarked +2560,0.4,0.2903688524590164,-7.987447429719284,0.004095133323667272,Watermarked +2561,0.6065573770491803,0.45833333333333337,-5.720862788355775,0.010598994876723512,Watermarked +2562,0.5573770491803278,0.3666666666666667,-5.605717509713631,0.01121846256816809,Watermarked +2563,0.55,0.3857923497267759,-2.773401460624423,0.06937128230425361,Watermarked +2564,0.5901639344262295,0.44583333333333336,-8.401275011504453,0.0035376591875335185,Watermarked +2565,0.6065573770491803,0.39583333333333337,-5.442971730211302,0.012177401351288936,Watermarked +2566,0.6229508196721312,0.41250000000000003,-4.643102959582609,0.018831812762783767,Watermarked +2567,0.5666666666666667,0.45614754098360655,-3.6102043743828824,0.03649838253815035,Watermarked +2568,0.65,0.4227459016393443,-5.2448185798443125,0.013495034751188689,Watermarked +2569,0.6229508196721312,0.425,-4.312990302477515,0.022955131152546198,Watermarked +2570,0.6229508196721312,0.4541666666666667,-6.130104668195612,0.008728726955771456,Watermarked +2571,0.6833333333333333,0.4685109289617486,-6.044103218158989,0.009083452194363606,Watermarked +2572,0.6666666666666666,0.551844262295082,-16.28766880858878,0.0005035373682223364,Watermarked +2573,0.6229508196721312,0.3416666666666667,-4.923541270396146,0.016055668617949045,Watermarked +2574,0.5,0.4148224043715847,-2.935331303657975,0.06073856617509204,Watermarked +2575,0.5666666666666667,0.39788251366120214,-3.995817683425296,0.028085239330913513,Watermarked +2576,0.5245901639344263,0.42083333333333334,-5.615165491162119,0.011165876393712151,Watermarked +2577,0.6166666666666667,0.44760928961748636,-2.8779672917258896,0.06363455777153253,Watermarked +2578,0.5737704918032787,0.4125,-4.800754235493788,0.017200753129888367,Watermarked +2579,0.5901639344262295,0.33749999999999997,-6.629461654119429,0.006991312298326812,Watermarked +2580,0.4918032786885246,0.4041666666666667,-10.98401471759805,0.0016157647819501492,Watermarked +2581,0.6,0.4559426229508197,-2.024940270686622,0.13600606005905447,Watermarked +2582,0.5166666666666667,0.36878415300546447,-2.8875369089248184,0.06313962819005703,Watermarked +2583,0.6,0.3898224043715847,-6.736784395106943,0.006678717888232159,Watermarked +2584,0.6721311475409836,0.4041666666666667,-15.59782388553334,0.0005726503128458851,Watermarked +2585,0.5409836065573771,0.35416666666666663,-10.11025517993969,0.002061088422236367,Watermarked +2586,0.5666666666666667,0.43135245901639346,-8.36023757475581,0.0035882787219289397,Watermarked +2587,0.5333333333333333,0.40683060109289615,-3.685355043522012,0.034628080098915076,Watermarked +2588,0.639344262295082,0.37083333333333335,-8.342713328952133,0.0036101866302316646,Watermarked +2589,0.6557377049180327,0.37916666666666665,-22.125683060109264,0.00020211389888430634,Watermarked +2590,0.5666666666666667,0.40204918032786885,-3.8720368025363827,0.030485627570044506,Watermarked +2591,0.5833333333333334,0.41509562841530057,-3.918936678185741,0.029546209898049555,Watermarked +2592,0.47540983606557374,0.37083333333333335,-2.2816691505216085,0.10677112400407492,Watermarked +2593,0.5666666666666667,0.4145491803278688,-3.225298064940852,0.04838862749274675,Watermarked +2594,0.6065573770491803,0.3833333333333333,-5.3934540699330284,0.012490325558165266,Watermarked +2595,0.65,0.3523224043715847,-6.072607039010736,0.008963822063323797,Watermarked +2596,0.6557377049180327,0.4125,-6.486338797814206,0.007438839601531942,Watermarked +2597,0.5,0.4363387978142077,-3.468352695270501,0.07401784897865982,Watermarked +2598,0.8333333333333334,0.4443306010928962,-13.069044138229902,0.0009675211431764483,Watermarked +2599,0.5737704918032787,0.41250000000000003,-3.6410524102519206,0.03571540330483068,Watermarked +2600,0.43333333333333335,0.194603825136612,-7.079275684498563,0.005796358119040252,Watermarked +2601,0.6721311475409836,0.2666666666666667,-13.671082075940005,0.0008467582451760971,Watermarked +2602,0.4426229508196721,0.3041666666666667,-3.470708937267535,0.04032563476307778,Watermarked +2603,0.4,0.2525273224043716,-2.709448750044224,0.07319856881186551,Watermarked +2604,0.4918032786885246,0.21666666666666665,-5.554402221351035,0.011509810078003106,Watermarked +2605,0.45,0.26939890710382514,-3.323461871958838,0.04493779701217889,Watermarked +2606,0.4918032786885246,0.24583333333333332,-7.477098647437464,0.004954385859733728,Watermarked +2607,0.5245901639344263,0.35416666666666663,-3.5645342704092955,0.03769829534729827,Watermarked +2608,0.38333333333333336,0.2655737704918033,-4.795356126981156,0.017253497348406005,Watermarked +2609,0.5737704918032787,0.2583333333333333,-8.132012676182216,0.0038879932294071462,Watermarked +2610,0.39344262295081966,0.21666666666666667,-9.819764009062963,0.0022448478763413245,Watermarked +2611,0.4426229508196721,0.26666666666666666,-2.7725030413659453,0.06942330291539806,Watermarked +2612,0.35,0.21147540983606558,-10.383401644446371,0.001906063349217342,Watermarked +2613,0.6885245901639344,0.19166666666666668,-22.535357185812448,0.00019134068124253585,Watermarked +2614,0.6557377049180327,0.20416666666666666,-12.92280294143108,0.0010002691021999664,Watermarked +2615,0.6229508196721312,0.25416666666666665,-11.210456983480942,0.001521591601458421,Watermarked +2616,0.5737704918032787,0.21666666666666667,-15.150632180505205,0.0006243223321137057,Watermarked +2617,0.5245901639344263,0.22499999999999998,-6.918738471764206,0.006189509895466707,Watermarked +2618,0.5166666666666667,0.2528688524590164,-8.617015849137902,0.0032865107935262,Watermarked +2619,0.4,0.23620218579234975,-5.140402367328491,0.014264695462063416,Watermarked +2620,0.5737704918032787,0.2791666666666667,-9.365095173805795,0.0025786301461165144,Watermarked +2621,0.5081967213114754,0.22916666666666663,-8.306260504867486,0.003656327731735171,Watermarked +2622,0.5409836065573771,0.2375,-7.720607509620097,0.004517414616084237,Watermarked +2623,0.7540983606557377,0.15833333333333333,-14.492905117377825,0.0007122141225715405,Watermarked +2624,0.4098360655737705,0.23333333333333334,-8.20310571032784,0.0037911953280657033,Watermarked +2625,0.4666666666666667,0.2568989071038251,-6.954617925271148,0.006098663645228381,Watermarked +2626,0.6721311475409836,0.22916666666666669,-17.17085230252723,0.00043034591455487535,Watermarked +2627,0.36666666666666664,0.2658469945355191,-2.8102104403444548,0.06728103964001445,Watermarked +2628,0.6333333333333333,0.5185109289617487,-7.578250897242925,0.004766379016909529,Watermarked +2629,0.75,0.19460382513661204,-10.85330590771855,0.0016736652012390987,Watermarked +2630,0.35,0.23203551912568304,-2.7592753489500113,0.07019486191792593,Watermarked +2631,0.48333333333333334,0.27404371584699455,-3.6739700755121674,0.03490341382115382,Watermarked +2632,0.4,0.2650273224043716,-2.3392495841230168,0.10129085666135956,Watermarked +2633,0.47540983606557374,0.2791666666666667,-4.2816691505216085,0.023404806469187092,Watermarked +2634,0.48333333333333334,0.24487704918032788,-16.399313267142833,0.0004934130615457107,Watermarked +2635,0.43333333333333335,0.25314207650273224,-4.511368522252938,0.02035188309309117,Watermarked +2636,0.6885245901639344,0.42500000000000004,-7.254802469447766,0.005403292565475987,Watermarked +2637,0.5833333333333334,0.2614071038251366,-6.303666365112512,0.008066415912528251,Watermarked +2638,0.45901639344262296,0.26249999999999996,-4.999367050456228,0.015397780557052671,Watermarked +2639,0.5,0.2284153005464481,-14.21620797979023,0.0007541140145429063,Watermarked +2640,0.6885245901639344,0.24166666666666667,-17.24695131862833,0.0004247197725637856,Watermarked +2641,0.43333333333333335,0.23210382513661199,-5.707259199146484,0.010669808386105378,Watermarked +2642,0.36666666666666664,0.24446721311475408,-3.2958071028221907,0.045877518615205004,Watermarked +2643,0.5737704918032787,0.2666666666666667,-11.653770786981173,0.0013573049832111713,Watermarked +2644,0.3770491803278688,0.24583333333333335,-4.076914857521508,0.02664334911823053,Watermarked +2645,0.4918032786885246,0.2333333333333333,-4.863758570369447,0.016600374637456124,Watermarked +2646,0.5901639344262295,0.2,-6.147724135202747,0.008658307683879244,Watermarked +2647,0.5409836065573771,0.23333333333333334,-17.08972839908021,0.00043645329683166667,Watermarked +2648,0.4426229508196721,0.2583333333333333,-6.667849243203331,0.006877333446979579,Watermarked +2649,0.47540983606557374,0.23333333333333334,-9.186157399702243,0.0027280043187776987,Watermarked +2650,0.5409836065573771,0.4375,-1.9737703484788938,0.14292112743640065,Watermarked +2651,0.5166666666666667,0.2614071038251366,-5.835105865583961,0.010027921838910365,Watermarked +2652,0.45,0.18640710382513662,-8.230490181795378,0.0037547595934809518,Watermarked +2653,0.5737704918032787,0.2583333333333333,-15.901213347642667,0.0005407927864783102,Watermarked +2654,0.5166666666666667,0.2823087431693989,-10.150428532833908,0.0020372645895073123,Watermarked +2655,0.5901639344262295,0.23750000000000002,-6.178438384650274,0.008537325917522763,Watermarked +2656,0.4426229508196721,0.27083333333333337,-3.3626410531598294,0.04364785672227368,Watermarked +2657,0.45,0.3445355191256831,-1.8802591489231106,0.15665493572520436,Watermarked +2658,0.47540983606557374,0.25416666666666665,-4.330653825520752,0.022706457712737285,Watermarked +2659,0.36065573770491804,0.30416666666666664,-3.0571045827459344,0.05511854521199261,Watermarked +2660,0.43333333333333335,0.24023224043715846,-3.433838233132664,0.0414217747379,Watermarked +2661,0.3770491803278688,0.24166666666666664,-2.7999079904008193,0.06785808116951951,Watermarked +2662,0.4666666666666667,0.2189207650273224,-2.8547072311071635,0.06485789803329635,Watermarked +2663,0.5409836065573771,0.4708333333333333,-2.930782830598391,0.06096209353124864,Watermarked +2664,0.5245901639344263,0.24166666666666664,-19.60151487691337,0.0002900998829147762,Watermarked +2665,0.45,0.27329234972677596,-2.6358380883941255,0.07793359764901263,Watermarked +2666,0.4666666666666667,0.2693306010928962,-5.264461390477094,0.013356347106673735,Watermarked +2667,0.6166666666666667,0.22418032786885245,-12.077933694787529,0.0012214555571584704,Watermarked +2668,0.48333333333333334,0.19467213114754098,-7.753219242030175,0.004462810515423161,Watermarked +2669,0.7049180327868853,0.4375,-4.557262825860137,0.01980483320019867,Watermarked +2670,0.8,0.3069672131147541,-32.249362363733205,6.552485880934655e-05,Watermarked +2671,0.4666666666666667,0.29836065573770487,-4.028754863050068,0.027487792515242023,Watermarked +2672,0.6229508196721312,0.21666666666666667,-24.377049180327866,0.0001513221713591045,Watermarked +2673,0.5,0.24870218579234973,-9.945483469303717,0.0021627595309605323,Watermarked +2674,0.48333333333333334,0.2364071038251366,-7.416787809974502,0.005071157125568877,Watermarked +2675,0.5573770491803278,0.25,-9.035003149610084,0.0028632366668939696,Watermarked +2676,0.48333333333333334,0.25724043715846995,-3.574953217198332,0.03742017108862267,Watermarked +2677,0.6,0.26967213114754096,-18.098951116688973,0.0003679240150022286,Watermarked +2678,0.45,0.273155737704918,-2.9827908163528103,0.05846639473064487,Watermarked +2679,0.3442622950819672,0.20416666666666666,-3.023496692892297,0.0566019195281106,Watermarked +2680,0.36666666666666664,0.25293715846994536,-3.3702819227387244,0.04340178717234347,Watermarked +2681,0.7377049180327869,0.19999999999999998,-17.24492210110332,0.00042486852229209587,Watermarked +2682,0.38333333333333336,0.2944672131147541,-2.8051586461529827,0.06756322441457491,Watermarked +2683,0.6557377049180327,0.14583333333333334,-17.025299295981913,0.00044138624174763324,Watermarked +2684,0.4426229508196721,0.275,-2.9340384355396165,0.060801997551779355,Watermarked +2685,0.4098360655737705,0.30833333333333335,-5.11676115950563,0.014446859559659811,Watermarked +2686,0.4166666666666667,0.2201502732240437,-3.2307527644980745,0.04818825006053326,Watermarked +2687,0.4262295081967213,0.2791666666666667,-10.333341919958322,0.0019332884778574828,Watermarked +2688,0.5,0.21577868852459017,-9.111666695534357,0.0027935663080181916,Watermarked +2689,0.4262295081967213,0.21250000000000002,-8.929327630907842,0.002963093223014625,Watermarked +2690,0.4918032786885246,0.4375,-2.2687169915537657,0.10805287324168507,Watermarked +2691,0.5737704918032787,0.2916666666666667,-6.359769644709016,0.007866504158921796,Watermarked +2692,0.39344262295081966,0.26666666666666666,-1.8012412407306406,0.1694691787567682,Watermarked +2693,0.6885245901639344,0.49166666666666675,-6.916091828453289,0.006196281549258644,Watermarked +2694,0.36666666666666664,0.2776639344262295,-1.7553554396997046,0.1774643858274755,Watermarked +2695,0.36065573770491804,0.2916666666666667,-1.899261403354651,0.15374343061615317,Watermarked +2696,0.4666666666666667,0.26584699453551913,-4.361077099679005,0.02228625751121354,Watermarked +2697,0.4262295081967213,0.27083333333333337,-3.735739626625215,0.03344241564305693,Watermarked +2698,0.47540983606557374,0.2375,-7.082180009413329,0.0057895528079337285,Watermarked +2699,0.4262295081967213,0.25416666666666665,-4.377269933993477,0.02206670433265803,Watermarked +2700,0.45901639344262296,0.24583333333333332,-5.194910461282009,0.013855966641037953,Watermarked +2701,0.5,0.28169398907103826,-5.477980736046312,0.011962325656887837,Watermarked +2702,0.55,0.2239754098360656,-10.180119908482158,0.0020198909534647554,Watermarked +2703,0.4918032786885246,0.2125,-4.076969624956957,0.0266424079833841,Watermarked +2704,0.55,0.3689890710382514,-4.353209549753575,0.022393951473013263,Watermarked +2705,0.4098360655737705,0.23333333333333334,-3.01552163240712,0.05696125979179388,Watermarked +2706,0.6557377049180327,0.20416666666666666,-18.1470510379736,0.0003650272002577954,Watermarked +2707,0.5737704918032787,0.2125,-8.356089793398965,0.003593448152596275,Watermarked +2708,0.6333333333333333,0.4937841530054645,-13.399578464312397,0.000898581498174531,Watermarked +2709,0.47540983606557374,0.22916666666666669,-5.559506115611259,0.01148039203424712,Watermarked +2710,0.55,0.2409153005464481,-5.275556807499205,0.013278830036287586,Watermarked +2711,0.39344262295081966,0.25,-4.714005617872331,0.018074445236289073,Watermarked +2712,0.5573770491803278,0.3041666666666667,-6.34727085863382,0.007910472276561024,Watermarked +2713,0.38333333333333336,0.2657103825136612,-3.2393534601694043,0.04787443221098791,Watermarked +2714,0.6166666666666667,0.4649590163934426,-6.9498273973418225,0.0061106911480797435,Watermarked +2715,0.8360655737704918,0.4375,-8.601708046331813,0.0033035386244482587,Watermarked +2716,0.5833333333333334,0.2448087431693989,-8.253674606606968,0.0037242731002333166,Watermarked +2717,0.35,0.24057377049180328,-5.639493439517238,0.011031947405375732,Watermarked +2718,0.5833333333333334,0.21530054644808744,-9.600064444773704,0.002398495294738282,Watermarked +2719,0.65,0.26140710382513666,-8.577234238877079,0.003331005620411765,Watermarked +2720,0.7704918032786885,0.43333333333333335,-18.728885297038175,0.00033227343416865454,Watermarked +2721,0.43333333333333335,0.27827868852459015,-3.3172720549176096,0.04514599422781829,Watermarked +2722,0.5245901639344263,0.2375,-10.122383386195422,0.0020538573428606206,Watermarked +2723,0.45901639344262296,0.3,-3.5639741636483593,0.037713321483800306,Watermarked +2724,0.36666666666666664,0.24480874316939888,-2.9309984722316638,0.06095147293868425,Watermarked +2725,0.4166666666666667,0.31912568306010924,-2.530970883389505,0.08535147636509026,Watermarked +2726,0.5166666666666667,0.24453551912568305,-6.751934642062348,0.006636082160327463,Watermarked +2727,0.3333333333333333,0.24890710382513662,-2.3788710766712677,0.14042438082786876,Watermarked +2728,0.5666666666666667,0.4769125683060109,-3.385054287067082,0.04293102665536962,Watermarked +2729,0.6166666666666667,0.24904371584699453,-16.87653237880174,0.00045306323873643635,Watermarked +2730,0.4426229508196721,0.25416666666666665,-4.94476472119911,0.01586789705539504,Watermarked +2731,0.39344262295081966,0.17916666666666667,-7.782330369943198,0.004414801539065831,Watermarked +2732,0.4666666666666667,0.29829234972677593,-3.6409046416348776,0.035719102507377344,Watermarked +2733,0.6065573770491803,0.18333333333333335,-31.10048861943559,7.303903972764834e-05,Watermarked +2734,0.5737704918032787,0.2,-9.42090076205074,0.0025342671020132644,Watermarked +2735,0.5245901639344263,0.22499999999999998,-5.482448602285279,0.011935236042507276,Watermarked +2736,0.6666666666666666,0.5806010928961749,-3.3948578066343797,0.04262218843494368,Watermarked +2737,0.5737704918032787,0.29166666666666663,-8.21042730655783,0.0037814081546606237,Watermarked +2738,0.32786885245901637,0.1958333333333333,-5.118158442680822,0.014436008902853572,Watermarked +2739,0.55,0.23210382513661199,-12.123135549004308,0.0012080613029538494,Watermarked +2740,0.7166666666666667,0.2116120218579235,-23.085068346256453,0.00017805380137088774,Watermarked +2741,0.4262295081967213,0.3,-2.9706771945521644,0.05903607413760807,Watermarked +2742,0.4262295081967213,0.23333333333333334,-2.8349832104998747,0.0659183394825102,Watermarked +2743,0.5,0.2941256830601093,-5.153821208882444,0.014162630156722995,Watermarked +2744,0.4262295081967213,0.24583333333333335,-2.8527267194908648,0.06496341216084892,Watermarked +2745,0.4166666666666667,0.2069672131147541,-3.211725828402651,0.048891793073391886,Watermarked +2746,0.4,0.2360655737704918,-3.510803607398461,0.03917521433599371,Watermarked +2747,0.38333333333333336,0.3026639344262295,-2.8694731663260846,0.06407793061054502,Watermarked +2748,0.5901639344262295,0.21666666666666667,-7.61224281479834,0.004705316919230158,Watermarked +2749,0.4666666666666667,0.2573087431693989,-5.082825548341707,0.014713685416969534,Watermarked +2750,0.48333333333333334,0.22862021857923498,-3.065403117058221,0.05475981670342542,Watermarked +2751,0.43333333333333335,0.26509562841530054,-2.866308813315449,0.06424408813835332,Watermarked +2752,0.45,0.30273224043715846,-5.035807306036759,0.015094095073797963,Watermarked +2753,0.65,0.3404371584699453,-6.119691979185821,0.008770696803910683,Watermarked +2754,0.5409836065573771,0.20416666666666666,-7.431074551672135,0.005043170304209105,Watermarked +2755,0.5081967213114754,0.25,-6.605727113313177,0.007063029498989654,Watermarked +2756,0.4166666666666667,0.2698087431693989,-9.036379043539897,0.0028619661292264213,Watermarked +2757,0.5081967213114754,0.2791666666666667,-7.116030982441239,0.005711005406184151,Watermarked +2758,0.45,0.28620218579234974,-3.720511834007275,0.033795204522967935,Watermarked +2759,0.4,0.2201502732240437,-5.295186309462342,0.013143123507478411,Watermarked +2760,0.6666666666666666,0.28217213114754097,-16.292961239551758,0.0005030512239375957,Watermarked +2761,0.47540983606557374,0.275,-3.8509508464272586,0.03092048438850596,Watermarked +2762,0.3770491803278688,0.24166666666666664,-6.140373651674497,0.008687594169327536,Watermarked +2763,0.6666666666666666,0.17397540983606558,-18.4422633103658,0.0003478966667273698,Watermarked +2764,0.65,0.4228142076502732,-4.415045976602256,0.02156528848562188,Watermarked +2765,0.7049180327868853,0.19999999999999998,-8.806809104164614,0.003084718357015228,Watermarked +2766,0.45,0.20314207650273225,-9.794313683048983,0.002261966026846209,Watermarked +2767,0.48333333333333334,0.2532103825136612,-5.1768582175628834,0.013989618848905717,Watermarked +2768,0.5833333333333334,0.2275273224043716,-5.726067308431419,0.010572065566569503,Watermarked +2769,0.639344262295082,0.1708333333333333,-12.10156988574371,0.0012144271996402047,Watermarked +2770,0.4098360655737705,0.23333333333333334,-5.951158369984666,0.00948839549552589,Watermarked +2771,0.47540983606557374,0.20416666666666666,-4.968520257961496,0.015661100563564314,Watermarked +2772,0.5333333333333333,0.2527322404371585,-4.948933058520406,0.01583135472899473,Watermarked +2773,0.47540983606557374,0.25,-4.9384820758662515,0.015923182580957953,Watermarked +2774,0.5833333333333334,0.2739071038251366,-13.930528616417067,0.0008008837712144284,Watermarked +2775,0.6229508196721312,0.15833333333333333,-7.384810731767565,0.005134544759237181,Watermarked +2776,0.38333333333333336,0.30273224043715846,-4.0696430409706865,0.026768690892997455,Watermarked +2777,0.48333333333333334,0.3855191256830601,-2.636692552502379,0.07787649597226969,Watermarked +2778,0.5901639344262295,0.22083333333333333,-8.148412107258412,0.003865377073276304,Watermarked +2779,0.45,0.298224043715847,-2.2807020090731873,0.10686619566510568,Watermarked +2780,0.4262295081967213,0.18333333333333332,-4.17817014100824,0.02497413039088068,Watermarked +2781,0.5333333333333333,0.27322404371584696,-5.040542454345847,0.015055208208642631,Watermarked +2782,0.4166666666666667,0.21530054644808744,-4.283002068724982,0.02338543904381472,Watermarked +2783,0.5081967213114754,0.19583333333333333,-7.410760977286471,0.005083024752688326,Watermarked +2784,0.3442622950819672,0.27499999999999997,-4.798632565231811,0.017221458650986843,Watermarked +2785,0.43333333333333335,0.2864071038251366,-4.100734975708682,0.02623802994170771,Watermarked +2786,0.47540983606557374,0.31666666666666665,-4.974054989029201,0.015613424561556911,Watermarked +2787,0.5,0.19863387978142077,-6.300594478470427,0.00807755369838508,Watermarked +2788,0.35,0.244603825136612,-2.2254146723514387,0.11247501852404906,Watermarked +2789,0.6229508196721312,0.2041666666666667,-13.312635870763579,0.0009160652152262239,Watermarked +2790,0.6833333333333333,0.4937158469945355,-8.885516696979227,0.003005843404969766,Watermarked +2791,0.45901639344262296,0.24166666666666664,-13.62086314551715,0.0008560385924757967,Watermarked +2792,0.4,0.26154371584699454,-3.4496293922245655,0.040947730245819165,Watermarked +2793,0.5166666666666667,0.29890710382513663,-3.8375524808634864,0.031200942303648546,Watermarked +2794,0.48333333333333334,0.2693306010928962,-6.246793566459261,0.008275938325843453,Watermarked +2795,0.4666666666666667,0.2655054644808743,-9.468017765185019,0.002497594914513529,Watermarked +2796,0.5166666666666667,0.19501366120218577,-6.412883435087076,0.007683202378919043,Watermarked +2797,0.4666666666666667,0.30683060109289617,-2.6055084852992985,0.07999436123101161,Watermarked +2798,0.5081967213114754,0.19999999999999998,-6.752251856457131,0.006635193291969306,Watermarked +2799,0.6229508196721312,0.20833333333333334,-19.77027395258561,0.0002827789468772716,Watermarked +2800,0.6557377049180327,0.4208333333333334,-10.236282712367263,0.0019875612473827316,Watermarked +2801,0.7049180327868853,0.4541666666666667,-4.489735890023653,0.020616535562147837,Watermarked +2802,0.6166666666666667,0.36038251366120216,-4.0670388186695,0.02681376369186907,Watermarked +2803,0.6833333333333333,0.4560792349726776,-5.9382601309311465,0.009546449129946996,Watermarked +2804,0.5245901639344263,0.425,-2.2720572719623497,0.10772054769411174,Watermarked +2805,0.7049180327868853,0.45,-3.6218924086243263,0.036199162722362586,Watermarked +2806,0.5666666666666667,0.38545081967213113,-5.020124422686412,0.015223827432133234,Watermarked +2807,0.5833333333333334,0.4937158469945355,-3.4194195863846306,0.041860729913222816,Watermarked +2808,0.5737704918032787,0.5,-2.260714065293725,0.10885413598684734,Watermarked +2809,0.7,0.3735655737704918,-10.454060628342173,0.0018684937780359754,Watermarked +2810,0.639344262295082,0.4,-9.756144412236056,0.0022879647959347044,Watermarked +2811,0.7377049180327869,0.39583333333333337,-7.4590163934426235,0.004989020225851709,Watermarked +2812,0.6333333333333333,0.3608606557377049,-12.95979435165334,0.0009918481137665518,Watermarked +2813,0.6065573770491803,0.37083333333333335,-10.272001058557553,0.001967356422200276,Watermarked +2814,0.7,0.435655737704918,-6.464633818447615,0.00750996829937376,Watermarked +2815,0.7166666666666667,0.45211748633879784,-15.725247417039386,0.0005589740150536343,Watermarked +2816,0.5901639344262295,0.4125,-3.7159734017373043,0.033901269284630456,Watermarked +2817,0.55,0.40191256830601096,-2.7474782254251675,0.07089198741296619,Watermarked +2818,0.6,0.45642076502732243,-10.118093819799428,0.0020564110187835205,Watermarked +2819,0.5166666666666667,0.39822404371584696,-2.815880143112684,0.06696608581805782,Watermarked +2820,0.639344262295082,0.48333333333333334,-5.404377109955353,0.012420402409580856,Watermarked +2821,0.5573770491803278,0.4083333333333333,-6.759987541853552,0.006613565506268803,Watermarked +2822,0.6721311475409836,0.4916666666666667,-6.965216353380622,0.006072164935164103,Watermarked +2823,0.5833333333333334,0.3817622950819672,-6.45646486744287,0.007536970061691775,Watermarked +2824,0.6229508196721312,0.425,-18.39984547099852,0.0003502913010235412,Watermarked +2825,0.6229508196721312,0.3583333333333334,-6.037002655406131,0.009113578699967435,Watermarked +2826,0.6065573770491803,0.37083333333333335,-8.311288381099306,0.003649917396478642,Watermarked +2827,0.6229508196721312,0.3833333333333333,-5.370455913263936,0.012639237313128257,Watermarked +2828,0.5166666666666667,0.44371584699453553,-2.834044674981603,0.06596933591930301,Watermarked +2829,0.6333333333333333,0.5101092896174864,-3.547608678177134,0.038155756341829127,Watermarked +2830,0.55,0.43558743169398906,-3.3200966453052976,0.045050836488169606,Watermarked +2831,0.6166666666666667,0.4814890710382514,-8.921688593482292,0.0029704891181838475,Watermarked +2832,0.6885245901639344,0.36249999999999993,-10.885700407196474,0.0016590620208720404,Watermarked +2833,0.5901639344262295,0.4625,-3.8807823860207193,0.030307571526173836,Watermarked +2834,0.6065573770491803,0.4041666666666667,-3.796834767979554,0.032073534237681285,Watermarked +2835,0.5573770491803278,0.35,-6.095615458270268,0.008868760073876481,Watermarked +2836,0.5833333333333334,0.4852459016393443,-3.2682313237305527,0.04683949096861523,Watermarked +2837,0.5081967213114754,0.4291666666666667,-3.443843878513792,0.041120604000996194,Watermarked +2838,0.5833333333333334,0.4435109289617486,-3.1918220004780653,0.04964171462252754,Watermarked +2839,0.6885245901639344,0.45833333333333337,-13.269636352208451,0.000924879437843544,Watermarked +2840,0.5833333333333334,0.3773224043715847,-4.587084180430928,0.01945956097411938,Watermarked +2841,0.6229508196721312,0.4666666666666667,-5.570797323763848,0.011415660012176308,Watermarked +2842,0.7049180327868853,0.4541666666666666,-3.965300941783783,0.028653776872905008,Watermarked +2843,0.7868852459016393,0.32916666666666666,-12.411848878312394,0.0011269497196247178,Watermarked +2844,0.7213114754098361,0.4291666666666667,-8.523579646154461,0.0033922878667983014,Watermarked +2845,0.5666666666666667,0.47704918032786886,-5.705858082917919,0.01067713705598062,Watermarked +2846,0.5573770491803278,0.36250000000000004,-4.348783893927762,0.022454827486827797,Watermarked +2847,0.6065573770491803,0.4708333333333333,-3.0287534515367915,0.05636661686037541,Watermarked +2848,0.5166666666666667,0.4523224043715847,-4.428901327226881,0.02138508188391737,Watermarked +2849,0.639344262295082,0.37916666666666665,-5.497766397039839,0.011842967860121692,Watermarked +2850,0.65,0.3694672131147541,-7.27109198627894,0.005368618417249257,Watermarked +2851,0.7049180327868853,0.35833333333333334,-8.672148663324567,0.003226136549403669,Watermarked +2852,0.5901639344262295,0.3625,-4.761760610530732,0.01758652689930633,Watermarked +2853,0.45,0.3647540983606557,-7.531019558041822,0.017178611116784117,Watermarked +2854,0.5081967213114754,0.4083333333333333,-3.1652931763476935,0.05066398901162397,Watermarked +2855,0.6065573770491803,0.3625,-6.508196721311476,0.007368102187221459,Watermarked +2856,0.5166666666666667,0.4562841530054645,-2.5241076101780897,0.08586662930511714,Watermarked +2857,0.7049180327868853,0.42916666666666664,-5.888028951637414,0.00977702506049218,Watermarked +2858,0.6,0.4227459016393443,-4.090863905829874,0.02640502342776308,Watermarked +2859,0.6333333333333333,0.38504098360655736,-3.331454908237819,0.044670745752969254,Watermarked +2860,0.5166666666666667,0.4189207650273224,-2.2659856668962592,0.10832553396730833,Watermarked +2861,0.6557377049180327,0.4125,-8.339578454332552,0.0036141243204565873,Watermarked +2862,0.5737704918032787,0.4083333333333333,-2.501174736689476,0.08761587052799776,Watermarked +2863,0.7166666666666667,0.40669398907103826,-7.514384861135337,0.004883963889775299,Watermarked +2864,0.6333333333333333,0.3781420765027323,-5.634742497670123,0.011057936145282518,Watermarked +2865,0.5,0.3776639344262295,-4.858609077032151,0.01664841190941072,Watermarked +2866,0.5573770491803278,0.4625,-2.1172312753822844,0.12450912024159524,Watermarked +2867,0.6557377049180327,0.43750000000000006,-9.957781870946098,0.002154944280259222,Watermarked +2868,0.6557377049180327,0.37916666666666665,-7.6307221538215995,0.004672553983451911,Watermarked +2869,0.6,0.4523224043715847,-6.124375580236181,0.00875178616778322,Watermarked +2870,0.5,0.34795081967213115,-2.822931024840667,0.06657696557341944,Watermarked +2871,0.7704918032786885,0.3208333333333333,-14.29409263370358,0.0007419958228994995,Watermarked +2872,0.6557377049180327,0.3416666666666667,-11.034070169059069,0.0015942956657695156,Watermarked +2873,0.55,0.4352459016393443,-2.8995067504928964,0.06252731441759059,Watermarked +2874,0.6,0.3688524590163934,-4.890622801652107,0.016352674036444928,Watermarked +2875,0.6065573770491803,0.36666666666666664,-4.753997634988252,0.01766466674069792,Watermarked +2876,0.5409836065573771,0.4375,-12.970212234731923,0.0009894935210338785,Watermarked +2877,0.5245901639344263,0.32916666666666666,-5.592518411047395,0.011292471025751814,Watermarked +2878,0.4918032786885246,0.39583333333333337,-4.8738225751540805,0.016507011615195296,Watermarked +2879,0.5081967213114754,0.43333333333333335,-3.1761845581166224,0.050241108410324004,Watermarked +2880,0.6557377049180327,0.43749999999999994,-3.491803278688525,0.039715098203703336,Watermarked +2881,0.5081967213114754,0.375,-3.0387597258755705,0.05592210337011148,Watermarked +2882,0.7377049180327869,0.4458333333333333,-8.515608096333422,0.0034015195584640207,Watermarked +2883,0.6666666666666666,0.45170765027322407,-4.285643375079597,0.02334712177086421,Watermarked +2884,0.48333333333333334,0.4147540983606558,-2.7228776982622653,0.07237357811230395,Watermarked +2885,0.65,0.46509562841530055,-5.141289506242194,0.014257918214039442,Watermarked +2886,0.47540983606557374,0.4208333333333334,-2.4902246368970027,0.08846651702809234,Watermarked +2887,0.5737704918032787,0.43333333333333335,-2.8622478469687973,0.06445811513039201,Watermarked +2888,0.6333333333333333,0.45157103825136613,-2.989953698734684,0.05813278125064246,Watermarked +2889,0.5573770491803278,0.3833333333333333,-3.057286949843683,0.05511063014547841,Watermarked +2890,0.6065573770491803,0.4166666666666667,-10.548278185933164,0.0018199105302834477,Watermarked +2891,0.6166666666666667,0.49337431693989064,-2.7692008371142,0.0696149257313345,Watermarked +2892,0.5333333333333333,0.3858606557377049,-3.299959459423778,0.0457348408632345,Watermarked +2893,0.5901639344262295,0.4625,-3.800342933561155,0.031997130689119066,Watermarked +2894,0.4666666666666667,0.3866120218579235,-1.7856892936312567,0.21607017204333234,Watermarked +2895,0.5737704918032787,0.4916666666666667,-3.723879479976154,0.0337167759198274,Watermarked +2896,0.5333333333333333,0.43948087431693994,-2.3754283562972685,0.09802174273100749,Watermarked +2897,0.5333333333333333,0.37725409836065577,-3.8749319798464548,0.03042653470881461,Watermarked +2898,0.639344262295082,0.4,-13.295383727881227,0.0009195881666894832,Watermarked +2899,0.5666666666666667,0.4316939890710383,-6.669993903142867,0.006871037958981735,Watermarked +2900,0.7049180327868853,0.4916666666666667,-9.672172825695142,0.0023465711838438533,Watermarked +2901,0.7049180327868853,0.4666666666666667,-9.041003671038775,0.0028577010889647217,Watermarked +2902,0.5666666666666667,0.4603825136612022,-3.7003174802745145,0.03427044013621927,Watermarked +2903,0.7666666666666667,0.3653005464480874,-8.367710625283678,0.0035789897552198605,Watermarked +2904,0.5245901639344263,0.3666666666666667,-11.604959601054722,0.0013742030324602752,Watermarked +2905,0.65,0.4771174863387978,-4.574500631486023,0.019604293549578128,Watermarked +2906,0.5409836065573771,0.39999999999999997,-4.521534453338067,0.020229034371625897,Watermarked +2907,0.5833333333333334,0.4359972677595628,-3.71598604315986,0.03390097325869757,Watermarked +2908,0.6065573770491803,0.37083333333333335,-4.12972900744298,0.025755332738221934,Watermarked +2909,0.7377049180327869,0.35416666666666663,-11.65895831862298,0.0013555253490377533,Watermarked +2910,0.6557377049180327,0.37916666666666665,-8.407323638843176,0.0035302782268764117,Watermarked +2911,0.5901639344262295,0.32499999999999996,-8.404697750055918,0.003533480017692563,Watermarked +2912,0.6333333333333333,0.47704918032786886,-3.572872262389389,0.037475510464252,Watermarked +2913,0.5409836065573771,0.4041666666666667,-3.3807925592909616,0.04306616942393178,Watermarked +2914,0.55,0.43558743169398906,-4.55116556463313,0.01987640371413144,Watermarked +2915,0.6885245901639344,0.5,-3.265341686400343,0.04694176800449449,Watermarked +2916,0.5333333333333333,0.42711748633879776,-3.8576108132230886,0.030782280046241853,Watermarked +2917,0.55,0.4313524590163934,-3.8365066246858697,0.03122297154382187,Watermarked +2918,0.6885245901639344,0.3458333333333333,-11.442187082108672,0.0014326132345143508,Watermarked +2919,0.5833333333333334,0.4420765027322404,-4.718756309978549,0.04209478615042156,Watermarked +2920,0.65,0.45266393442622954,-5.549004690060408,0.011541028000440172,Watermarked +2921,0.6065573770491803,0.4666666666666667,-4.2869837090014355,0.02332770866411145,Watermarked +2922,0.5573770491803278,0.4291666666666667,-3.7406497535262835,0.03332966691975382,Watermarked +2923,0.7333333333333333,0.5189890710382514,-7.795621704107285,0.004393108665011521,Watermarked +2924,0.6166666666666667,0.3902322404371585,-7.19301070709552,0.005537540195779914,Watermarked +2925,0.5833333333333334,0.360724043715847,-7.579568529572813,0.004763992639250445,Watermarked +2926,0.7333333333333333,0.3858606557377049,-14.089106567555575,0.0007744612617815288,Watermarked +2927,0.5573770491803278,0.4041666666666667,-2.527395473158105,0.08561936927249675,Watermarked +2928,0.7377049180327869,0.3875,-12.007025761124119,0.001242867214816079,Watermarked +2929,0.5333333333333333,0.4230874316939891,-3.347260874883563,0.04414856066329664,Watermarked +2930,0.7540983606557377,0.35000000000000003,-16.471846559014978,0.00048698021922821526,Watermarked +2931,0.7049180327868853,0.38749999999999996,-39.783878716039034,3.4943182204626624e-05,Watermarked +2932,0.7049180327868853,0.3833333333333333,-27.287366605133304,0.0001080165038157327,Watermarked +2933,0.5833333333333334,0.3605874316939891,-4.472129156189748,0.020835236225830767,Watermarked +2934,0.5409836065573771,0.42083333333333334,-6.50235424552611,0.007386922898311131,Watermarked +2935,0.7,0.36502732240437163,-7.572061952950856,0.004777609010065385,Watermarked +2936,0.6721311475409836,0.425,-4.356743170503809,0.022345498848743366,Watermarked +2937,0.5333333333333333,0.4153688524590164,-3.1703304743958,0.05046784910179293,Watermarked +2938,0.48333333333333334,0.3284153005464481,-3.071040072795025,0.05451781694215867,Watermarked +2939,0.5901639344262295,0.4375,-5.097326714872012,0.014598887274167346,Watermarked +2940,0.43333333333333335,0.39781420765027325,-16.249999999999922,0.0037656052779771634,Watermarked +2941,0.5,0.35947176684881604,-3.9829601606847738,0.05763976433444822,Watermarked +2942,0.5,0.4313524590163934,-1.9917554271299593,0.14044440711390466,Watermarked +2943,0.5666666666666667,0.41905737704918034,-7.182256405781872,0.005561352568453646,Watermarked +2944,0.5166666666666667,0.3982923497267759,-5.599507245339936,0.011253204715190618,Watermarked +2945,0.5333333333333333,0.39829234972677596,-4.723689577243244,0.017974094449026225,Watermarked +2946,0.5901639344262295,0.38333333333333336,-4.2565386541927595,0.02377386602678355,Watermarked +2947,0.6229508196721312,0.42499999999999993,-3.2030019695384855,0.04921870916506516,Watermarked +2948,0.45901639344262296,0.3875,-1.7671941136183904,0.1753608911595616,Watermarked +2949,0.6721311475409836,0.42500000000000004,-5.444705021331377,0.01216663456000868,Watermarked +2950,0.5573770491803278,0.3375,-4.229541871678541,0.024178754668284275,Watermarked +2951,0.65,0.48087431693989074,-2.870168301345979,0.06404150157384371,Watermarked +2952,0.639344262295082,0.3958333333333333,-8.130635229762294,0.003889900792911817,Watermarked +2953,0.5901639344262295,0.4541666666666667,-4.048412402739574,0.02713900943382794,Watermarked +2954,0.5666666666666667,0.4519808743169399,-2.7715039029613733,0.0694812123937475,Watermarked +2955,0.6333333333333333,0.4562841530054645,-6.639086391609726,0.006962502866864333,Watermarked +2956,0.6,0.3402322404371585,-5.379181269835038,0.012582469390043725,Watermarked +2957,0.6557377049180327,0.4125,-3.2184309034451113,0.04864239224720965,Watermarked +2958,0.5833333333333334,0.3690573770491803,-6.0781372785179055,0.008940851611301535,Watermarked +2959,0.6229508196721312,0.37916666666666665,-11.123418935025548,0.0015569047785033382,Watermarked +2960,0.6557377049180327,0.38749999999999996,-8.52694846917119,0.0033883964608289563,Watermarked +2961,0.6166666666666667,0.4227459016393443,-3.8587298371444883,0.030759136480267034,Watermarked +2962,0.5666666666666667,0.4936475409836066,-2.5087837781411735,0.08703067245521756,Watermarked +2963,0.7377049180327869,0.39583333333333337,-21.672082554883993,0.00021500571596787181,Watermarked +2964,0.5666666666666667,0.4441256830601093,-4.114953099885994,0.025999880249744926,Watermarked +2965,0.7,0.3152322404371585,-27.094781124739157,0.00011032862635579921,Watermarked +2966,0.819672131147541,0.375,-9.79685729377205,0.002260247404737733,Watermarked +2967,0.4918032786885246,0.325,-4.952772918130208,0.015797788968726655,Watermarked +2968,0.6557377049180327,0.4083333333333333,-7.3460047131317,0.00521288063379473,Watermarked +2969,0.45,0.3919854280510018,-1.2592474439830368,0.33499662016654086,Watermarked +2970,0.639344262295082,0.41250000000000003,-6.491699980107792,0.00742140717977802,Watermarked +2971,0.6557377049180327,0.39583333333333337,-16.47597884542129,0.0004866171022863723,Watermarked +2972,0.6833333333333333,0.4605874316939891,-9.319488874116459,0.002615651895897316,Watermarked +2973,0.6666666666666666,0.37725409836065577,-9.746825292040107,0.0022943725355199732,Watermarked +2974,0.6166666666666667,0.4519125683060109,-4.467667765204196,0.020891129917487965,Watermarked +2975,0.639344262295082,0.3833333333333333,-3.8203179931250597,0.03156651788894358,Watermarked +2976,0.7540983606557377,0.41250000000000003,-6.306431273644387,0.008056408388901619,Watermarked +2977,0.7166666666666667,0.4853142076502732,-5.664156791225322,0.010898310561203092,Watermarked +2978,0.6166666666666667,0.35239071038251363,-10.603077839669773,0.001792421776010322,Watermarked +2979,0.5333333333333333,0.42288251366120216,-2.792068808043707,0.06830128777723389,Watermarked +2980,0.5573770491803278,0.3666666666666667,-6.797930999977556,0.00650881472279479,Watermarked +2981,0.5573770491803278,0.4416666666666667,-2.8952739132187233,0.06274299207471501,Watermarked +2982,0.55,0.4230191256830601,-3.5968393291702223,0.03684441767090311,Watermarked +2983,0.5833333333333334,0.36495901639344264,-8.629044905647506,0.003273211490827028,Watermarked +2984,0.5245901639344263,0.4041666666666667,-2.9757050833898977,0.05879878128106479,Watermarked +2985,0.7377049180327869,0.4666666666666667,-15.055959655317555,0.0006360495232655051,Watermarked +2986,0.5737704918032787,0.42083333333333334,-3.1359127797177497,0.05182736123156866,Watermarked +2987,0.7377049180327869,0.36666666666666664,-12.510329824209249,0.0011009408982364437,Watermarked +2988,0.65,0.4400273224043716,-4.731321598151042,0.017895516622468247,Watermarked +2989,0.6721311475409836,0.41666666666666663,-8.004718441897252,0.004069631315123374,Watermarked +2990,0.6885245901639344,0.37083333333333335,-10.34387657107953,0.0019275165468191863,Watermarked +2991,0.55,0.4271174863387978,-4.102891976310035,0.026201720340923294,Watermarked +2992,0.5166666666666667,0.4565573770491803,-3.871284033511624,0.030501016317830993,Watermarked +2993,0.6229508196721312,0.375,-7.362230013436795,0.005179936955568627,Watermarked +2994,0.5081967213114754,0.4041666666666667,-5.629951970401516,0.01108422263908312,Watermarked +2995,0.6666666666666666,0.38989071038251366,-7.027749672525287,0.005918864007946058,Watermarked +2996,0.6,0.4684426229508197,-3.0715294425556663,0.05449687138415423,Watermarked +2997,0.55,0.3942622950819672,-5.155433917254585,0.01415042791070789,Watermarked +2998,0.5166666666666667,0.4093806921675774,-2.1811187398383804,0.16093929016287928,Watermarked +2999,0.5666666666666667,0.4518442622950819,-2.7116536407871275,0.07306231680161718,Watermarked +3000,0.5333333333333333,0.38995901639344266,-4.717870559277373,0.01803430740365783,Watermarked +3001,0.5333333333333333,0.33572404371584696,-5.92922902548398,0.009587374714266027,Watermarked +3002,0.7540983606557377,0.31666666666666665,-8.09966109145276,0.0039331240116504485,Watermarked +3003,0.5,0.3855191256830601,-2.7763365971020724,0.06920166849483687,Watermarked +3004,0.7049180327868853,0.35833333333333334,-13.376800824017138,0.0009031187936782386,Watermarked +3005,0.4666666666666667,0.3734972677595629,-2.323557226050761,0.10274990807505914,Watermarked +3006,0.6166666666666667,0.4319672131147541,-3.959212335659781,0.028768976615567175,Watermarked +3007,0.5333333333333333,0.3937841530054645,-3.3540166875307085,0.04392773109730455,Watermarked +3008,0.6666666666666666,0.3860655737704918,-6.951550947054901,0.006106360263638599,Watermarked +3009,0.7704918032786885,0.3875,-11.40102880068347,0.0014479028367797866,Watermarked +3010,0.6333333333333333,0.43572404371584705,-11.348123445291295,0.0014678760475146785,Watermarked +3011,0.7213114754098361,0.2875,-27.500378882331333,0.00010553371875927762,Watermarked +3012,0.5333333333333333,0.30683060109289617,-10.923824320280426,0.0016420917950501913,Watermarked +3013,0.5409836065573771,0.3125,-3.893753295658278,0.03004594161105454,Watermarked +3014,0.6666666666666666,0.3523907103825137,-6.8536427559842705,0.006358933335598355,Watermarked +3015,0.6166666666666667,0.4937158469945355,-4.691276871564279,0.018312849057309264,Watermarked +3016,0.6557377049180327,0.33333333333333337,-5.788824671837737,0.010254290938715958,Watermarked +3017,0.55,0.3273224043715847,-5.436414841751896,0.012218244055546734,Watermarked +3018,0.4918032786885246,0.3333333333333333,-3.361655189247521,0.043679734758324804,Watermarked +3019,0.5245901639344263,0.37916666666666665,-2.9532447616676007,0.059868156846439195,Watermarked +3020,0.5737704918032787,0.43333333333333335,-4.127992550132961,0.025783917599408773,Watermarked +3021,0.5166666666666667,0.3692622950819672,-8.104386708891786,0.003926488707999252,Watermarked +3022,0.5245901639344263,0.3666666666666667,-4.309973795297764,0.022997950137582556,Watermarked +3023,0.5166666666666667,0.3651639344262295,-3.2122265752036507,0.04887311160481489,Watermarked +3024,0.5666666666666667,0.39002732240437155,-3.9886831470208737,0.02821684888437735,Watermarked +3025,0.5901639344262295,0.3458333333333333,-4.2504183551249355,0.02386488383717604,Watermarked +3026,0.6229508196721312,0.32499999999999996,-6.880879437718984,0.006287310969453765,Watermarked +3027,0.5666666666666667,0.4148907103825137,-6.273699658520553,0.008175934849579894,Watermarked +3028,0.5081967213114754,0.375,-2.754702614168314,0.07046406236659389,Watermarked +3029,0.4918032786885246,0.3958333333333333,-6.7433095204558455,0.006660310850275737,Watermarked +3030,0.5737704918032787,0.38749999999999996,-4.020027050959187,0.02764449995869553,Watermarked +3031,0.55,0.4400273224043716,-2.9245926757956453,0.061267955766122074,Watermarked +3032,0.6885245901639344,0.375,-7.524590163934425,0.004864919741574715,Watermarked +3033,0.7333333333333333,0.28647540983606556,-23.29092393225503,0.00017339475386319076,Watermarked +3034,0.5081967213114754,0.3458333333333333,-3.5424739195231005,0.038295935976556825,Watermarked +3035,0.55,0.36099726775956287,-6.160098807321899,0.008609295466603118,Watermarked +3036,0.5,0.4400956284153006,-2.282897070641292,0.10665056460060143,Watermarked +3037,0.55,0.41994535519125686,-18.636835301251896,0.0028667184146571527,Watermarked +3038,0.5833333333333334,0.4271174863387978,-5.673540790894663,0.010848020755982598,Watermarked +3039,0.5333333333333333,0.44405737704918036,-3.2047054428325206,0.04915465698343356,Watermarked +3040,0.7377049180327869,0.3625,-11.657726867446977,0.0013559475300634065,Watermarked +3041,0.5333333333333333,0.36919398907103823,-6.079597077972364,0.008934801065512471,Watermarked +3042,0.7704918032786885,0.3416666666666666,-39.859982700803556,3.47437157745963e-05,Watermarked +3043,0.6065573770491803,0.33749999999999997,-9.2248243559719,0.0026947741822498272,Watermarked +3044,0.7,0.4937158469945355,-8.461896046771958,0.0034645967635944996,Watermarked +3045,0.6333333333333333,0.33551912568306014,-6.233747163298384,0.008325007314823952,Watermarked +3046,0.5409836065573771,0.36249999999999993,-12.541115855481232,0.0010929740797637178,Watermarked +3047,0.5901639344262295,0.39166666666666666,-3.4744562728924717,0.040216306514409425,Watermarked +3048,0.5166666666666667,0.40204918032786885,-2.291468930292345,0.10581353287285375,Watermarked +3049,0.6229508196721312,0.42500000000000004,-3.9774734463808428,0.028425239599828966,Watermarked +3050,0.5833333333333334,0.41898907103825134,-7.392993166333208,0.005118225808922169,Watermarked +3051,0.48333333333333334,0.42732240437158475,-2.616902813780725,0.07921236839961415,Watermarked +3052,0.5666666666666667,0.323224043715847,-6.786089292025601,0.006541270274250083,Watermarked +3053,0.6,0.3608606557377049,-6.358986733603966,0.007869248834482912,Watermarked +3054,0.7,0.32786885245901637,-15.380260404150008,0.0005970483655978608,Watermarked +3055,0.6,0.44849726775956283,-4.876103332248839,0.016485948236315353,Watermarked +3056,0.4166666666666667,0.3813296903460838,-3.099243212836256,0.09023900265155212,Watermarked +3057,0.5333333333333333,0.3650956284153006,-4.977491357685013,0.015583918885552142,Watermarked +3058,0.55,0.38545081967213113,-2.575853873389877,0.08207463601397176,Watermarked +3059,0.639344262295082,0.4,-5.945876360497398,0.009512113138117317,Watermarked +3060,0.6833333333333333,0.37739071038251365,-5.80205523533122,0.010188900334164743,Watermarked +3061,0.45901639344262296,0.375,-2.8708139379727218,0.06400768969988004,Watermarked +3062,0.6,0.4312158469945355,-3.2417038925561115,0.04778912095848229,Watermarked +3063,0.6,0.36933060109289617,-6.847157062045659,0.006376146501783941,Watermarked +3064,0.5166666666666667,0.36912568306010923,-7.1062884927858425,0.005733467077920565,Watermarked +3065,0.639344262295082,0.4,-11.123716102549825,0.0015567823708010134,Watermarked +3066,0.6557377049180327,0.4166666666666667,-2.859337167599795,0.06461206589437947,Watermarked +3067,0.6166666666666667,0.41475409836065574,-8.016759438310023,0.004051975840158878,Watermarked +3068,0.5409836065573771,0.37083333333333335,-3.8415339067926904,0.03111726238521636,Watermarked +3069,0.6666666666666666,0.35273224043715845,-8.96185148037276,0.0029318759099350045,Watermarked +3070,0.6833333333333333,0.3729508196721312,-6.271729950256445,0.008183201757294712,Watermarked +3071,0.5081967213114754,0.3416666666666667,-3.845846875093736,0.031026938964603432,Watermarked +3072,0.48333333333333334,0.3443306010928962,-6.819969086862238,0.006448975326342902,Watermarked +3073,0.5245901639344263,0.4083333333333333,-2.753689634807033,0.0705238707366333,Watermarked +3074,0.5166666666666667,0.3471766848816029,-2.090326535903569,0.1717476912950698,Watermarked +3075,0.639344262295082,0.33333333333333337,-6.424891128611615,0.007642542451366619,Watermarked +3076,0.5,0.37745901639344265,-5.307134884333113,0.013061404240156596,Watermarked +3077,0.5166666666666667,0.38558743169398907,-3.5605909993218026,0.037804245228875336,Watermarked +3078,0.7704918032786885,0.30833333333333335,-13.194558267956465,0.0009405413630643912,Watermarked +3079,0.7540983606557377,0.3625,-11.206538990050577,0.001523158238254348,Watermarked +3080,0.47540983606557374,0.3416666666666666,-2.7938036914857083,0.06820289228132356,Watermarked +3081,0.5666666666666667,0.3652322404371584,-6.1214946474517395,0.008763411957595306,Watermarked +3082,0.6721311475409836,0.38333333333333336,-8.021244634914645,0.004045425154564468,Watermarked +3083,0.5,0.423224043715847,-3.689180746726386,0.03453618347086469,Watermarked +3084,0.5737704918032787,0.42500000000000004,-3.2776582466721216,0.04650778639185666,Watermarked +3085,0.5081967213114754,0.3666666666666667,-2.5040961733848097,0.08739061882540976,Watermarked +3086,0.7868852459016393,0.3666666666666667,-10.75090825425996,0.0017209617612538612,Watermarked +3087,0.5409836065573771,0.45833333333333337,-9.918032786885249,0.002180339575209168,Watermarked +3088,0.7540983606557377,0.3083333333333333,-27.93518318279224,0.0001006969515148238,Watermarked +3089,0.5333333333333333,0.3562841530054645,-3.0165050287504305,0.05691679534461103,Watermarked +3090,0.5245901639344263,0.4,-2.792392128219332,0.06828293698877178,Watermarked +3091,0.5,0.3187841530054645,-3.0923105332523986,0.053616690038148854,Watermarked +3092,0.6065573770491803,0.35833333333333334,-5.034902560136769,0.015101540087103106,Watermarked +3093,0.7377049180327869,0.3416666666666667,-6.611700289223251,0.0070448899240306145,Watermarked +3094,0.48333333333333334,0.3692622950819672,-3.8314981973665483,0.03132874291483428,Watermarked +3095,0.5901639344262295,0.35,-4.446971954645207,0.021152969420364667,Watermarked +3096,0.5666666666666667,0.4068306010928962,-5.623397158172275,0.011120322780836269,Watermarked +3097,0.5833333333333334,0.37684426229508194,-3.3673616030988653,0.04349562581061357,Watermarked +3098,0.4918032786885246,0.36250000000000004,-3.567536931880534,0.03761787225198254,Watermarked +3099,0.6833333333333333,0.33599726775956285,-19.586542860762346,0.0002907615152797266,Watermarked +3100,0.75,0.3402322404371585,-98.69856142441458,2.2928614096026574e-06,Watermarked +3101,0.43333333333333335,0.34763205828779603,-2.164602989862948,0.16283491133527353,Watermarked +3102,0.7540983606557377,0.4083333333333333,-13.34516471643176,0.0009094713400266177,Watermarked +3103,0.5833333333333334,0.3486338797814208,-4.5541089442634854,0.019841812100798096,Watermarked +3104,0.6721311475409836,0.325,-10.30712201881151,0.0019477541958685205,Watermarked +3105,0.4426229508196721,0.3416666666666666,-9.384048173269779,0.0025634488134986455,Watermarked +3106,0.5333333333333333,0.3570355191256831,-5.484975321134636,0.011919951523226141,Watermarked +3107,0.6,0.4395491803278689,-6.233752412361234,0.00832498749577228,Watermarked +3108,0.5166666666666667,0.33613387978142073,-5.360731285926958,0.012702901609225676,Watermarked +3109,0.6166666666666667,0.4563524590163935,-3.2403169660324136,0.04783943753394908,Watermarked +3110,0.7049180327868853,0.3875,-20.121911614894287,0.0002682965121865791,Watermarked +3111,0.5901639344262295,0.4083333333333333,-9.166095697580069,0.002745459142872881,Watermarked +3112,0.6721311475409836,0.35833333333333334,-5.742446164819555,0.010487900478810956,Watermarked +3113,0.6721311475409836,0.3666666666666667,-7.815029537231881,0.004361685111408271,Watermarked +3114,0.6229508196721312,0.44999999999999996,-3.1053610315875337,0.05307307725713686,Watermarked +3115,0.55,0.36454918032786887,-3.2455626872934524,0.04764947889276525,Watermarked +3116,0.5333333333333333,0.37356557377049177,-3.757035055298464,0.032956925528189414,Watermarked +3117,0.639344262295082,0.39999999999999997,-5.493611809608042,0.011867900939411315,Watermarked +3118,0.5833333333333334,0.3897540983606558,-7.467934583305662,0.004971898730909863,Watermarked +3119,0.5,0.42342896174863387,-4.800324631602227,0.01720494304786845,Watermarked +3120,0.5737704918032787,0.33749999999999997,-14.977768942526218,0.0006459569455958266,Watermarked +3121,0.5166666666666667,0.29849726775956287,-4.379387604175447,0.022038198857139507,Watermarked +3122,0.4918032786885246,0.39166666666666666,-3.42163677485204,0.041792847188906634,Watermarked +3123,0.7213114754098361,0.35833333333333334,-16.463141062057648,0.0004877463789961118,Watermarked +3124,0.6833333333333333,0.3688524590163934,-7.8974364585461565,0.00423151561991115,Watermarked +3125,0.5901639344262295,0.37083333333333335,-3.8425267583017693,0.03109643993412762,Watermarked +3126,0.6065573770491803,0.3916666666666667,-4.682079879777176,0.01841047771126608,Watermarked +3127,0.6333333333333333,0.4646174863387978,-5.6726426012368485,0.010852821078198268,Watermarked +3128,0.4426229508196721,0.31666666666666665,-3.324802624172649,0.04489286075866494,Watermarked +3129,0.5409836065573771,0.31666666666666665,-5.419860209810303,0.012322162365242495,Watermarked +3130,0.5833333333333334,0.36045081967213116,-4.440891852865153,0.02123070084724967,Watermarked +3131,0.6166666666666667,0.3941256830601093,-3.223851145752677,0.04844195656899026,Watermarked +3132,0.6065573770491803,0.45833333333333337,-6.722810708442813,0.00671836273975336,Watermarked +3133,0.55,0.41461748633879786,-4.609601498177781,0.01920401476029972,Watermarked +3134,0.45,0.3482923497267759,-3.935890584039202,0.029215786175393728,Watermarked +3135,0.6721311475409836,0.35,-14.97130283665096,0.0006467854175943383,Watermarked +3136,0.7540983606557377,0.3541666666666667,-16.708601239682135,0.00046674362746457226,Watermarked +3137,0.5409836065573771,0.37916666666666665,-2.2740071535885997,0.10752712396614864,Watermarked +3138,0.6333333333333333,0.43907103825136606,-2.820281518337947,0.06672285377972131,Watermarked +3139,0.6885245901639344,0.3625,-3.419275812902317,0.04186513660401589,Watermarked +3140,0.6885245901639344,0.35833333333333334,-6.456069857786826,0.007538278964860644,Watermarked +3141,0.5081967213114754,0.4291666666666667,-1.8998904386836808,0.15364813434857005,Watermarked +3142,0.5901639344262295,0.44999999999999996,-3.585966147227154,0.03712903167578977,Watermarked +3143,0.6065573770491803,0.4291666666666667,-4.10299965843394,0.02619990938534542,Watermarked +3144,0.5737704918032787,0.37916666666666665,-5.466397186258622,0.01203293462437204,Watermarked +3145,0.48333333333333334,0.3855191256830601,-3.236738871470941,0.047969557784140165,Watermarked +3146,0.6666666666666666,0.38982240437158466,-7.304853191112693,0.005297682541612248,Watermarked +3147,0.8360655737704918,0.34583333333333327,-17.804840632796406,0.00038631936941714036,Watermarked +3148,0.5901639344262295,0.3708333333333333,-5.272729644436734,0.01329852584746764,Watermarked +3149,0.4666666666666667,0.3483606557377049,-4.11223200027819,0.026045241625233456,Watermarked +3150,0.4918032786885246,0.38750000000000007,-4.357648863022137,0.022333101933145045,Watermarked +3151,0.639344262295082,0.32916666666666666,-6.767511177347243,0.006592619506710128,Watermarked +3152,0.5333333333333333,0.4186475409836065,-2.434445731879166,0.0929600077713279,Watermarked +3153,0.5409836065573771,0.36666666666666664,-5.1238506750677635,0.0143919153374869,Watermarked +3154,0.5666666666666667,0.41072404371584703,-10.34274767901109,0.0019281339723313756,Watermarked +3155,0.48333333333333334,0.35280054644808745,-9.808946709611767,0.0022521026539224693,Watermarked +3156,0.5333333333333333,0.43531420765027323,-2.293238524273031,0.10564172479014122,Watermarked +3157,0.5333333333333333,0.35273224043715845,-8.189147517586767,0.0038099469501537706,Watermarked +3158,0.6333333333333333,0.36898907103825135,-9.328181302832693,0.0026085416180980188,Watermarked +3159,0.5901639344262295,0.375,-3.3240845596280026,0.04491692010565002,Watermarked +3160,0.5333333333333333,0.38162568306010936,-5.53218920098128,0.011638997397509718,Watermarked +3161,0.5166666666666667,0.4438524590163935,-4.609409203433426,0.01920617859490499,Watermarked +3162,0.5573770491803278,0.35416666666666663,-3.463054126930755,0.040550141822148895,Watermarked +3163,0.55,0.3030054644808743,-3.9665722337998846,0.028629798248878405,Watermarked +3164,0.6333333333333333,0.37336065573770494,-8.550441690064975,0.003361422362271298,Watermarked +3165,0.4426229508196721,0.375,-4.685055463096139,0.01837881723814625,Watermarked +3166,0.6166666666666667,0.31127049180327865,-9.651006540622726,0.002361656973312143,Watermarked +3167,0.7166666666666667,0.38599726775956283,-15.465401308907632,0.0005873388348340652,Watermarked +3168,0.6333333333333333,0.2568306010928962,-10.419231568279397,0.0018868885475834876,Watermarked +3169,0.8032786885245902,0.37916666666666665,-8.376268479684917,0.0035683913912364033,Watermarked +3170,0.5166666666666667,0.3978825136612022,-2.035753205981991,0.1345958069792505,Watermarked +3171,0.6,0.3980874316939891,-7.600738322488857,0.004725867063471558,Watermarked +3172,0.7868852459016393,0.39166666666666666,-27.381546373206323,0.00010690928977807902,Watermarked +3173,0.38333333333333336,0.3278005464480874,-5.343021351720979,0.01281992383399354,Watermarked +3174,0.5833333333333334,0.41857923497267757,-3.3484587546107205,0.0441093025667818,Watermarked +3175,0.5166666666666667,0.33989071038251367,-4.377554579883222,0.02206287000445764,Watermarked +3176,0.5,0.39207650273224043,-6.083165423794337,0.025975157859805906,Watermarked +3177,0.7704918032786885,0.3208333333333333,-15.854306407643215,0.0005455614883016697,Watermarked +3178,0.5,0.28189890710382515,-8.369667988980005,0.003576562016483774,Watermarked +3179,0.5833333333333334,0.3903005464480874,-6.562635068330882,0.0071957310516421215,Watermarked +3180,0.6666666666666666,0.3730191256830601,-6.867274869817217,0.006322952397605397,Watermarked +3181,0.43333333333333335,0.32568306010928966,-4.191489361702128,0.052479362028526676,Watermarked +3182,0.6721311475409836,0.35000000000000003,-6.5653506061794005,0.0071872722304183664,Watermarked +3183,0.7,0.3524590163934426,-10.447778102751764,0.0018717942236663328,Watermarked +3184,0.6557377049180327,0.36249999999999993,-70.37704918032789,6.3221094755592405e-06,Watermarked +3185,0.5166666666666667,0.45621584699453555,-2.481511075867372,0.0891506613942868,Watermarked +3186,0.5666666666666667,0.4271174863387978,-3.4482809420111318,0.040987939768970645,Watermarked +3187,0.5901639344262295,0.3666666666666667,-23.226517386743268,0.000174834804970807,Watermarked +3188,0.4918032786885246,0.39583333333333337,-7.677595628415292,0.004590789402025528,Watermarked +3189,0.5166666666666667,0.3941256830601093,-3.7613782647305922,0.03285902039196703,Watermarked +3190,0.6229508196721312,0.3416666666666666,-13.4125186386734,0.0008960173086668277,Watermarked +3191,0.5333333333333333,0.3273224043715847,-3.665214293431724,0.0351170707372584,Watermarked +3192,0.6333333333333333,0.52698087431694,-2.6670409685182177,0.07588150981935955,Watermarked +3193,0.6557377049180327,0.37916666666666665,-22.125683060109264,0.00020211389888430634,Watermarked +3194,0.5166666666666667,0.3484289617486339,-2.8306761052244935,0.06615277677623232,Watermarked +3195,0.48333333333333334,0.3771857923497267,-2.618745744139447,0.07908677590411582,Watermarked +3196,0.5166666666666667,0.3983606557377049,-3.9828547448399347,0.02832495388510772,Watermarked +3197,0.45901639344262296,0.3708333333333333,-2.3515482695810577,0.10016489140788615,Watermarked +3198,0.5166666666666667,0.3691256830601093,-7.521674420991118,0.004870350805489641,Watermarked +3199,0.6,0.3650956284153005,-41.98171043120708,2.974431577617351e-05,Watermarked +3200,0.5333333333333333,0.3980191256830601,-3.2824011075555344,0.04634202450941883,Watermarked +3201,0.39344262295081966,0.35416666666666663,-3.142076502732241,0.051580525827867194,Watermarked +3202,0.6065573770491803,0.4041666666666667,-11.78086978660089,0.0013145795310678767,Watermarked +3203,0.639344262295082,0.38333333333333336,-8.021840834286337,0.004044555451710541,Watermarked +3204,0.6666666666666666,0.332103825136612,-15.850883671738368,0.0005459116401288157,Watermarked +3205,0.48333333333333334,0.37329234972677594,-3.6587848750156016,0.03527502614042613,Watermarked +3206,0.36666666666666664,0.29863387978142075,-2.4701740765596973,0.09005051324793042,Watermarked +3207,0.7213114754098361,0.3666666666666667,-8.810214059275827,0.0030812498509113165,Watermarked +3208,0.4666666666666667,0.4105874316939891,-1.5057124818059264,0.22921714041813998,Watermarked +3209,0.5409836065573771,0.22083333333333333,-6.909375426419404,0.0062135098686104855,Watermarked +3210,0.5333333333333333,0.41509562841530057,-6.219125997443245,0.00838045426704938,Watermarked +3211,0.55,0.38995901639344266,-5.020631036144959,0.015219613947787137,Watermarked +3212,0.5245901639344263,0.3041666666666667,-5.783528726099863,0.010280619668641678,Watermarked +3213,0.5166666666666667,0.34378415300546444,-3.106070198750089,0.05304373654077345,Watermarked +3214,0.5,0.34433060109289615,-4.489204827567853,0.020623088486061487,Watermarked +3215,0.45901639344262296,0.29166666666666663,-15.55542491509537,0.000577299561388648,Watermarked +3216,0.5333333333333333,0.3945355191256831,-2.804910878658928,0.0675771021782272,Watermarked +3217,0.5833333333333334,0.3105191256830601,-4.772347925976209,0.017480680770223765,Watermarked +3218,0.7377049180327869,0.3375,-19.209836065573764,0.0003080910681863436,Watermarked +3219,0.5737704918032787,0.3833333333333333,-4.318708814548055,0.022874240057608075,Watermarked +3220,0.55,0.3239754098360656,-5.08778567271395,0.014674286692950959,Watermarked +3221,0.5333333333333333,0.3812158469945355,-2.9098120190283097,0.06200608865891807,Watermarked +3222,0.35,0.2569672131147541,-3.7827690734466466,0.03238222739381364,Watermarked +3223,0.7377049180327869,0.2875,-16.3510804808083,0.0004977534397666609,Watermarked +3224,0.4426229508196721,0.41250000000000003,-1.9095681132275377,0.15219065910307153,Watermarked +3225,0.5573770491803278,0.4041666666666667,-4.560818191534335,0.019763254051599725,Watermarked +3226,0.7377049180327869,0.35416666666666663,-13.929800773742766,0.0008010077875496463,Watermarked +3227,0.639344262295082,0.4083333333333333,-7.498639437160465,0.004913539903160214,Watermarked +3228,0.43333333333333335,0.36919398907103823,-2.8757941403931446,0.06374762528542408,Watermarked +3229,0.4666666666666667,0.28169398907103826,-3.4441072444884826,0.04111271435550431,Watermarked +3230,0.7166666666666667,0.36133879781420764,-10.501771543064775,0.0018436795460142676,Watermarked +3231,0.6557377049180327,0.32916666666666666,-20.7021431957843,0.00024648403432591493,Watermarked +3232,0.6666666666666666,0.3896857923497268,-7.435895472354905,0.005033772413915748,Watermarked +3233,0.6721311475409836,0.3541666666666667,-5.824341929189313,0.01007998096990893,Watermarked +3234,0.5333333333333333,0.37752732240437153,-6.922734884968511,0.0061793032033680305,Watermarked +3235,0.7377049180327869,0.41666666666666663,-9.080332982450201,0.002821767606719206,Watermarked +3236,0.55,0.38155737704918036,-3.5810708052340003,0.03725808735441134,Watermarked +3237,0.5245901639344263,0.37916666666666665,-4.242857392252246,0.02397794903308461,Watermarked +3238,0.6557377049180327,0.3625,-6.133284818784408,0.008715961470550591,Watermarked +3239,0.5,0.29494535519125686,-3.242117432574401,0.04777413102072636,Watermarked +3240,0.5081967213114754,0.38749999999999996,-2.474836034730356,0.08967913937363099,Watermarked +3241,0.5,0.37363387978142076,-3.3164206595199035,0.04517472690275175,Watermarked +3242,0.6666666666666666,0.4066256830601093,-18.30718715197799,0.0003555990078420392,Watermarked +3243,0.6721311475409836,0.3375,-11.473067915690864,0.0014212819594170815,Watermarked +3244,0.55,0.31058743169398906,-4.4611858896576715,0.020972683531277587,Watermarked +3245,0.5166666666666667,0.4523224043715847,-3.6924428701682177,0.034458070848123835,Watermarked +3246,0.8524590163934426,0.3541666666666667,-20.02461614309747,0.00027220292398855736,Watermarked +3247,0.6885245901639344,0.3416666666666667,-15.731996671646355,0.0005582617929085287,Watermarked +3248,0.5833333333333334,0.4019808743169399,-3.3737593587415304,0.04329038177941591,Watermarked +3249,0.4666666666666667,0.36092896174863387,-4.924844065432519,0.016044059231596925,Watermarked +3250,0.5901639344262295,0.3666666666666667,-4.511917593907792,0.020345223247711267,Watermarked +3251,0.45,0.37759562841530053,-3.483025090320139,0.039967726509492464,Watermarked +3252,0.6885245901639344,0.35833333333333334,-14.9760677256279,0.0006461747777244088,Watermarked +3253,0.47540983606557374,0.35,-4.022062624190254,0.027607848701928988,Watermarked +3254,0.6166666666666667,0.47691256830601103,-3.022304945858794,0.0566554359290112,Watermarked +3255,0.4666666666666667,0.3237704918032787,-4.585573326772214,0.019476865011623285,Watermarked +3256,0.7377049180327869,0.37916666666666665,-6.896820529493841,0.006245884135338378,Watermarked +3257,0.5245901639344263,0.38749999999999996,-2.4730386527896364,0.0898220971946934,Watermarked +3258,0.48333333333333334,0.373155737704918,-3.7564520938598576,0.03297009515071729,Watermarked +3259,0.43333333333333335,0.3278005464480874,-5.076865217882905,0.014761211511120415,Watermarked +3260,0.48333333333333334,0.31086065573770494,-2.446306172576996,0.0919815932829521,Watermarked +3261,0.6065573770491803,0.35833333333333334,-7.2243808310002775,0.005468843291212087,Watermarked +3262,0.5166666666666667,0.35252732240437157,-4.378773516969542,0.022046460036578352,Watermarked +3263,0.48333333333333334,0.37329234972677594,-2.5720050130375935,0.08234949684139341,Watermarked +3264,0.6721311475409836,0.36666666666666664,-5.484666087385246,0.011921820751493945,Watermarked +3265,0.65,0.3280054644808743,-14.159199715025988,0.0007631512846613169,Watermarked +3266,0.7213114754098361,0.35416666666666663,-6.623110492647877,0.007010408974038976,Watermarked +3267,0.5409836065573771,0.35833333333333334,-4.326302046694481,0.022767398907893536,Watermarked +3268,0.65,0.332103825136612,-10.547062196561422,0.0018205268185872753,Watermarked +3269,0.5737704918032787,0.32916666666666666,-4.41253184635759,0.021598198496940495,Watermarked +3270,0.7213114754098361,0.29166666666666663,-26.924962031836227,0.00011242244042797529,Watermarked +3271,0.5333333333333333,0.3239071038251366,-6.101922495764141,0.008842933069258837,Watermarked +3272,0.6666666666666666,0.36448087431693993,-5.276046667403815,0.013275421234137245,Watermarked +3273,0.48333333333333334,0.3691256830601093,-2.611497078469241,0.07958218244247245,Watermarked +3274,0.6333333333333333,0.3605191256830601,-7.086477957097316,0.0057795013156923015,Watermarked +3275,0.6666666666666666,0.3151639344262295,-6.220646342838296,0.008374666202957234,Watermarked +3276,0.6557377049180327,0.4125,-4.719504703575856,0.01801737147532206,Watermarked +3277,0.5245901639344263,0.30000000000000004,-4.334146739196724,0.022657696393709254,Watermarked +3278,0.5409836065573771,0.4,-2.990713926985726,0.05809751373410958,Watermarked +3279,0.5081967213114754,0.30416666666666664,-5.440801457194902,0.012190900140945336,Watermarked +3280,0.55,0.3399590163934426,-4.334577337946331,0.022651694585833295,Watermarked +3281,0.4666666666666667,0.3108606557377049,-2.3956053387350558,0.0962542828406371,Watermarked +3282,0.5166666666666667,0.4145491803278689,-2.833861411958958,0.0659792994727473,Watermarked +3283,0.5737704918032787,0.22916666666666669,-8.901067221690772,0.0029905765083464166,Watermarked +3284,0.4426229508196721,0.33749999999999997,-2.4314657198035565,0.09320783628411884,Watermarked +3285,0.5,0.27329234972677596,-4.855988855019134,0.016672924204011234,Watermarked +3286,0.6,0.25341530054644806,-13.539511357945873,0.0008713612072207959,Watermarked +3287,0.7049180327868853,0.3208333333333333,-11.206006160936766,0.0015233714589831572,Watermarked +3288,0.639344262295082,0.3416666666666667,-3.5780996247983943,0.037336695763226686,Watermarked +3289,0.639344262295082,0.29583333333333334,-13.315684846477943,0.0009154444688870522,Watermarked +3290,0.6557377049180327,0.3875,-11.688828790772387,0.0013453380246460723,Watermarked +3291,0.5666666666666667,0.3316939890710382,-5.9679306695524,0.009413595633304362,Watermarked +3292,0.5737704918032787,0.4,-3.3534274040500196,0.04394693731715465,Watermarked +3293,0.5666666666666667,0.3896857923497268,-3.9226085660313683,0.029474240628051502,Watermarked +3294,0.5409836065573771,0.3708333333333333,-2.9809200108781417,0.058553923379340976,Watermarked +3295,0.48333333333333334,0.3403005464480874,-2.952162851632012,0.05992028137117902,Watermarked +3296,0.7166666666666667,0.2653688524590164,-16.038628368958843,0.0005271380369548524,Watermarked +3297,0.4262295081967213,0.3083333333333333,-3.8269224330393166,0.03142577924163457,Watermarked +3298,0.5666666666666667,0.4144125683060109,-3.4211013109598394,0.041809228433188725,Watermarked +3299,0.639344262295082,0.32916666666666666,-4.849227812278989,0.016736391339350645,Watermarked +3300,0.5,0.3152322404371585,-17.69183502777838,0.00039371274155216776,Watermarked +3301,0.5901639344262295,0.35833333333333334,-11.68659984883048,0.001346094689122379,Watermarked +3302,0.5409836065573771,0.4,-2.873384998124319,0.06387326471495713,Watermarked +3303,0.48333333333333334,0.4063524590163934,-1.9862461743452617,0.14119767735260078,Watermarked +3304,0.5666666666666667,0.31898907103825136,-5.827473612902438,0.01006479830694792,Watermarked +3305,0.5166666666666667,0.3939207650273224,-3.0276117316398676,0.05641761781506673,Watermarked +3306,0.48333333333333334,0.3358606557377049,-4.734425236222974,0.017863689846178848,Watermarked +3307,0.6166666666666667,0.3656420765027322,-5.128769703376435,0.014353952607178531,Watermarked +3308,0.48333333333333334,0.3566939890710382,-7.883059676687519,0.004253852045425947,Watermarked +3309,0.6721311475409836,0.3416666666666667,-9.434713700912411,0.0025234427155974874,Watermarked +3310,0.6065573770491803,0.3041666666666667,-10.096584056706222,0.0020692799372390666,Watermarked +3311,0.4918032786885246,0.43333333333333335,-3.508196721311475,0.0392487316994922,Watermarked +3312,0.5409836065573771,0.35,-4.813749731486455,0.01707463310186129,Watermarked +3313,0.5245901639344263,0.29166666666666663,-7.216870606867377,0.005485187056437409,Watermarked +3314,0.5737704918032787,0.3375,-6.518811541138283,0.007334069087384167,Watermarked +3315,0.5409836065573771,0.22499999999999998,-7.68678962084268,0.004574973719624272,Watermarked +3316,0.6885245901639344,0.39166666666666666,-9.409277800552035,0.0025434230249447316,Watermarked +3317,0.5245901639344263,0.3541666666666667,-21.360184505383586,0.00022451203065569885,Watermarked +3318,0.47540983606557374,0.2125,-6.081034149138327,0.008928849971299894,Watermarked +3319,0.5333333333333333,0.3603142076502732,-2.9539865796064393,0.05983245019134663,Watermarked +3320,0.6885245901639344,0.33333333333333337,-8.952610373293831,0.0029407013393304916,Watermarked +3321,0.5,0.39849726775956285,-6.830325200725934,0.00642110575024065,Watermarked +3322,0.5245901639344263,0.4,-3.737704918032786,0.03339722912081489,Watermarked +3323,0.47540983606557374,0.2791666666666667,-11.423030339825978,0.00143970284382488,Watermarked +3324,0.65,0.39419398907103825,-4.965110212591529,0.01569056902102536,Watermarked +3325,0.55,0.3604508196721311,-3.3681723780241897,0.043469547365275946,Watermarked +3326,0.5573770491803278,0.375,-3.3374687113774946,0.04447114763467144,Watermarked +3327,0.65,0.34002732240437156,-9.8986749455488,0.0021928507964472117,Watermarked +3328,0.5081967213114754,0.4041666666666667,-3.473480352013456,0.040244742555193926,Watermarked +3329,0.5166666666666667,0.33189890710382514,-10.490828508285672,0.001849332165580751,Watermarked +3330,0.6666666666666666,0.3065573770491803,-8.898873673993194,0.002992723821402328,Watermarked +3331,0.6885245901639344,0.2708333333333333,-11.732895329579321,0.0013304938857159276,Watermarked +3332,0.7049180327868853,0.4,-10.562670498616757,0.0018126371846850279,Watermarked +3333,0.48333333333333334,0.4628415300546448,-5.357142857142865,0.11748359353956558,Watermarked +3334,0.5666666666666667,0.37336065573770494,-4.6106699178095605,0.019191997918002076,Watermarked +3335,0.5333333333333333,0.3980191256830601,-2.348482665780014,0.10044412446307645,Watermarked +3336,0.6557377049180327,0.37083333333333335,-5.4804035870677845,0.011947625433148852,Watermarked +3337,0.47540983606557374,0.3333333333333333,-3.1843068128816943,0.04992864292949258,Watermarked +3338,0.6557377049180327,0.4083333333333333,-4.59932645244697,0.01932008137256461,Watermarked +3339,0.639344262295082,0.35833333333333334,-10.845912217186935,0.0016770220983140147,Watermarked +3340,0.4918032786885246,0.2833333333333333,-5.896420481369791,0.009738001288109581,Watermarked +3341,0.6166666666666667,0.3812158469945356,-3.7975738512734774,0.032057418473780255,Watermarked +3342,0.6721311475409836,0.4291666666666667,-30.452174863987732,7.779154477118327e-05,Watermarked +3343,0.5573770491803278,0.22499999999999998,-23.027757458005887,0.00017938051259614428,Watermarked +3344,0.4426229508196721,0.29583333333333334,-3.202682563338301,0.04923073087668093,Watermarked +3345,0.6666666666666666,0.4817622950819672,-4.70814887470562,0.018135489628524907,Watermarked +3346,0.6721311475409836,0.3583333333333333,-9.722669668020147,0.0023110927865077316,Watermarked +3347,0.5573770491803278,0.32083333333333336,-4.437542626227876,0.021273677300139238,Watermarked +3348,0.6833333333333333,0.4314890710382514,-9.676632078012846,0.0023434092465502462,Watermarked +3349,0.5081967213114754,0.3208333333333333,-3.604111013983922,0.03665562896226882,Watermarked +3350,0.7704918032786885,0.3208333333333333,-21.583606557377053,0.00021764701504013405,Watermarked +3351,0.5901639344262295,0.41666666666666663,-3.2125451549484705,0.04886123097830394,Watermarked +3352,0.5573770491803278,0.3208333333333333,-8.591077485213033,0.0033154323787321117,Watermarked +3353,0.5333333333333333,0.38545081967213113,-2.9267563949002504,0.06116082615925972,Watermarked +3354,0.4666666666666667,0.3236338797814207,-13.636558407121642,0.000853123683257044,Watermarked +3355,0.6166666666666667,0.36516393442622946,-8.46907603191196,0.0034560758139621692,Watermarked +3356,0.5,0.36065573770491804,-3.1934829897182784,0.04957857968168316,Watermarked +3357,0.6,0.3235655737704918,-28.52854660180471,9.456145255444976e-05,Watermarked +3358,0.5409836065573771,0.4291666666666667,-2.0643127364438847,0.13095346493752535,Watermarked +3359,0.45,0.38148907103825136,-1.9632892320372335,0.1443881711602152,Watermarked +3360,0.65,0.3028005464480874,-21.112622602464683,0.00023245973293610858,Watermarked +3361,0.5081967213114754,0.37083333333333335,-3.54807654763531,0.038143015996558144,Watermarked +3362,0.5333333333333333,0.38155737704918036,-4.0869440858831725,0.026471716786619498,Watermarked +3363,0.6166666666666667,0.3941256830601093,-5.534321299197799,0.011626515414883042,Watermarked +3364,0.5081967213114754,0.29583333333333334,-9.689747679740668,0.002334141978546938,Watermarked +3365,0.45901639344262296,0.32083333333333336,-2.9505777976622465,0.059996749206154006,Watermarked +3366,0.5737704918032787,0.3375,-3.780327868852458,0.03243619070712851,Watermarked +3367,0.5573770491803278,0.375,-3.8488075064770464,0.03096513128016824,Watermarked +3368,0.45901639344262296,0.3416666666666667,-4.878136536726772,0.01646720054964036,Watermarked +3369,0.4666666666666667,0.3066256830601093,-3.3297523730648932,0.044727459212230945,Watermarked +3370,0.43333333333333335,0.35252732240437157,-2.5701626159312543,0.08248147008467596,Watermarked +3371,0.639344262295082,0.36666666666666664,-5.403746742275941,0.012424423736674949,Watermarked +3372,0.6166666666666667,0.41857923497267757,-2.837173206379461,0.06579953452896789,Watermarked +3373,0.43333333333333335,0.2981785063752277,-9.104294478527612,0.011850419592490571,Watermarked +3374,0.55,0.3736338797814208,-3.6168425013226613,0.03632805703920555,Watermarked +3375,0.7377049180327869,0.3458333333333333,-9.297077569799361,0.0026341027851884773,Watermarked +3376,0.5901639344262295,0.4041666666666667,-3.0118665582406012,0.05712690752613409,Watermarked +3377,0.45,0.3567622950819672,-2.922322679137377,0.061380599938098745,Watermarked +3378,0.5666666666666667,0.3567622950819672,-8.560154259731128,0.003350353702966942,Watermarked +3379,0.4426229508196721,0.3958333333333333,-2.1349234997177082,0.12243934625158044,Watermarked +3380,0.5833333333333334,0.3274590163934426,-7.792967812252545,0.004397428835019784,Watermarked +3381,0.4262295081967213,0.35833333333333334,-2.3199911010934016,0.10308502717775488,Watermarked +3382,0.6166666666666667,0.38155737704918036,-4.19677462214937,0.02468223424934266,Watermarked +3383,0.5901639344262295,0.35833333333333334,-9.637017116211096,0.002371698207192692,Watermarked +3384,0.5,0.28724954462659374,-20.478625837697965,0.0023760107227863043,Watermarked +3385,0.6333333333333333,0.39419398907103825,-23.001886718608972,0.00017998371795611563,Watermarked +3386,0.639344262295082,0.32916666666666666,-6.6941484555943065,0.006800654936853843,Watermarked +3387,0.7049180327868853,0.37916666666666665,-10.355237394824906,0.001921317535318784,Watermarked +3388,0.55,0.4269808743169399,-3.4667216700978054,0.04044237885506118,Watermarked +3389,0.5,0.3236338797814208,-3.3358653417919237,0.04452425266445473,Watermarked +3390,0.45,0.28565573770491803,-2.3900038580000214,0.0967410413896404,Watermarked +3391,0.7,0.3323087431693989,-10.546963974844278,0.001820576611444715,Watermarked +3392,0.5737704918032787,0.39166666666666666,-12.616523095570006,0.0010737810472414555,Watermarked +3393,0.4166666666666667,0.331511839708561,-2.9439949370436143,0.09860789571567317,Watermarked +3394,0.819672131147541,0.325,-13.385460331466769,0.0009013902444265222,Watermarked +3395,0.48333333333333334,0.19508196721311474,-6.738703907451196,0.00667329610025673,Watermarked +3396,0.7213114754098361,0.31666666666666665,-9.776873289560362,0.0022737966110181968,Watermarked +3397,0.5333333333333333,0.2782103825136612,-6.842713959859326,0.006387974012842435,Watermarked +3398,0.6833333333333333,0.31058743169398906,-7.0022122373829845,0.005980846181537664,Watermarked +3399,0.65,0.2774590163934426,-7.817198672088378,0.004358191490211607,Watermarked +3400,0.6065573770491803,0.37916666666666665,-3.7510920776385794,0.033091499306512666,Watermarked +3401,0.5245901639344263,0.41666666666666663,-5.0158308909899345,0.015259597669989054,Watermarked +3402,0.5,0.3396174863387978,-3.020014584938689,0.05675846509350086,Watermarked +3403,0.6065573770491803,0.35416666666666663,-6.420806830530328,0.0076563407117634,Watermarked +3404,0.55,0.42704918032786887,-3.2041931426214503,0.04917390886551068,Watermarked +3405,0.45,0.38995901639344266,-3.129964263210284,0.0520669925513877,Watermarked +3406,0.7,0.44002732240437165,-7.029706101430858,0.005914150507692506,Watermarked +3407,0.4166666666666667,0.3360655737704918,-6.662521026761018,0.006893006911496739,Watermarked +3408,0.6,0.41120218579234974,-4.744558099428093,0.01776029146137704,Watermarked +3409,0.7377049180327869,0.3916666666666667,-11.23243861284888,0.0015128416306198192,Watermarked +3410,0.65,0.4360655737704918,-4.896859345148878,0.01629585933682827,Watermarked +3411,0.5,0.41065573770491803,-3.340359673429475,0.04437559930810233,Watermarked +3412,0.5737704918032787,0.4083333333333333,-3.970491803278687,0.02855603110389197,Watermarked +3413,0.639344262295082,0.375,-4.954036545154963,0.015786763366299392,Watermarked +3414,0.6,0.439275956284153,-2.5509371831068104,0.0838742637314232,Watermarked +3415,0.6721311475409836,0.37083333333333335,-12.587812138042343,0.001081035521070716,Watermarked +3416,0.6721311475409836,0.42083333333333334,-13.599864280438059,0.000859959220110807,Watermarked +3417,0.6166666666666667,0.49330601092896176,-2.4421197772396783,0.09232550216594695,Watermarked +3418,0.43333333333333335,0.3809653916211293,-1.5855147752916146,0.25372930058134713,Watermarked +3419,0.7213114754098361,0.32083333333333336,-18.273075151598423,0.0003575799805904938,Watermarked +3420,0.6,0.4062841530054645,-6.551255287390875,0.007231321491454494,Watermarked +3421,0.5409836065573771,0.425,-3.0865517657726187,0.05385879831750464,Watermarked +3422,0.819672131147541,0.3875,-15.69614416660204,0.000562059046312007,Watermarked +3423,0.65,0.4394808743169399,-6.100988988496524,0.008846749505079436,Watermarked +3424,0.6833333333333333,0.4354508196721312,-8.022565688606122,0.004043498406787921,Watermarked +3425,0.639344262295082,0.45833333333333337,-7.5244830164877445,0.004865119180322468,Watermarked +3426,0.5333333333333333,0.3650273224043715,-11.968876772692552,0.0012545933074113616,Watermarked +3427,0.6885245901639344,0.32916666666666666,-6.2514540023238165,0.008258502009868794,Watermarked +3428,0.5166666666666667,0.38176229508196724,-9.8479960675198,0.0022260589594432483,Watermarked +3429,0.5333333333333333,0.40710382513661203,-3.2566855663164747,0.047249843238845575,Watermarked +3430,0.5333333333333333,0.44781420765027324,-2.777298311206574,0.0691462057873957,Watermarked +3431,0.639344262295082,0.38749999999999996,-4.973957462489661,0.015614263014698614,Watermarked +3432,0.5,0.4034608378870674,-5.908250072050177,0.027472183908806763,Watermarked +3433,0.6333333333333333,0.41933060109289616,-6.341832402564622,0.007929704462389898,Watermarked +3434,0.5333333333333333,0.4229508196721311,-4.809523809523813,0.017115512915415635,Watermarked +3435,0.43333333333333335,0.3479052823315118,-5.862500000000004,0.02788479615419901,Watermarked +3436,0.5901639344262295,0.44166666666666665,-6.172915501073344,0.008558915834601189,Watermarked +3437,0.6065573770491803,0.425,-2.836410863828114,0.06584086048198345,Watermarked +3438,0.5666666666666667,0.40607923497267756,-3.337367049030468,0.044474512395329985,Watermarked +3439,0.7377049180327869,0.3125,-11.156636603647863,0.0015433006224139084,Watermarked +3440,0.5409836065573771,0.39999999999999997,-2.3612943568800553,0.09928340428129473,Watermarked +3441,0.48333333333333334,0.3940573770491803,-7.029354227468431,0.005914997890496912,Watermarked +3442,0.5833333333333334,0.4853142076502732,-3.218376091512211,0.04864442449363043,Watermarked +3443,0.4,0.3277322404371585,-7.5308054776588165,0.004853369315234331,Watermarked +3444,0.55,0.4401639344262295,-2.2348801205435334,0.11149003001722149,Watermarked +3445,0.7704918032786885,0.38333333333333336,-9.35437505152399,0.0025872696994375557,Watermarked +3446,0.4666666666666667,0.35273224043715845,-2.771421867769551,0.06948596977369918,Watermarked +3447,0.6065573770491803,0.3916666666666667,-10.246669168835966,0.0019816576756549577,Watermarked +3448,0.6,0.3605191256830601,-5.027382355038384,0.015163608282268311,Watermarked +3449,0.5166666666666667,0.4354508196721312,-2.2311148149182007,0.11188060441585682,Watermarked +3450,0.55,0.385655737704918,-7.0347669547746365,0.005901980635791814,Watermarked +3451,0.4918032786885246,0.38749999999999996,-2.929866094565433,0.06100726967849578,Watermarked +3452,0.7213114754098361,0.35,-20.62605763263766,0.0002492064346505477,Watermarked +3453,0.5409836065573771,0.375,-3.46728445878972,0.040425874782353266,Watermarked +3454,0.7049180327868853,0.3833333333333333,-5.862266729790863,0.00989811735639771,Watermarked +3455,0.5409836065573771,0.3875,-4.478018111973132,0.02076175387541808,Watermarked +3456,0.48333333333333334,0.40225409836065573,-3.1045263606689577,0.05310763670073003,Watermarked +3457,0.55,0.39371584699453555,-2.5092989392309506,0.08699122587945884,Watermarked +3458,0.6885245901639344,0.3458333333333333,-7.3958473467775,0.005112549560971972,Watermarked +3459,0.45901639344262296,0.39999999999999997,-2.7428341074780405,0.07116877964790087,Watermarked +3460,0.6065573770491803,0.4083333333333333,-3.7454630727011047,0.03321961329360707,Watermarked +3461,0.55,0.3815573770491803,-5.86945467255433,0.009864134442699606,Watermarked +3462,0.5409836065573771,0.35,-2.4617916243617888,0.09072301062363927,Watermarked +3463,0.5245901639344263,0.3666666666666667,-4.8396027027399,0.016827287331982445,Watermarked +3464,0.5166666666666667,0.38155737704918036,-4.355025558660401,0.02236903359688658,Watermarked +3465,0.47540983606557374,0.39999999999999997,-3.694312398951678,0.03441340625590583,Watermarked +3466,0.6229508196721312,0.375,-8.04851009961339,0.004005903148525875,Watermarked +3467,0.5245901639344263,0.4166666666666667,-3.9653624934399807,0.028652615314166956,Watermarked +3468,0.5409836065573771,0.3541666666666667,-5.938681961594909,0.009544543156865669,Watermarked +3469,0.6557377049180327,0.4,-3.9840670207259876,0.028302424799336666,Watermarked +3470,0.6333333333333333,0.3528688524590164,-12.365301190960757,0.001139527243227325,Watermarked +3471,0.5081967213114754,0.37083333333333335,-7.995723638492874,0.004082886586878434,Watermarked +3472,0.6229508196721312,0.5458333333333334,-3.518727928731812,0.03895281371803606,Watermarked +3473,0.6166666666666667,0.3894125683060109,-4.405715742078374,0.021687749143792793,Watermarked +3474,0.5409836065573771,0.36250000000000004,-3.7715052757850085,0.032632179601749925,Watermarked +3475,0.5,0.360724043715847,-4.619815242020446,0.01908953586981084,Watermarked +3476,0.55,0.41475409836065574,-5.369818568286147,0.012643397039975352,Watermarked +3477,0.6833333333333333,0.4059426229508197,-4.851990369531891,0.016710420799139213,Watermarked +3478,0.5666666666666667,0.4314207650273224,-5.6998934006029405,0.01070840976220116,Watermarked +3479,0.55,0.45642076502732243,-8.972601513715789,0.0029216534818719115,Watermarked +3480,0.7704918032786885,0.3208333333333333,-14.29409263370358,0.0007419958228994995,Watermarked +3481,0.7213114754098361,0.4166666666666667,-9.335911788157421,0.002602239684135932,Watermarked +3482,0.55,0.3610655737704918,-3.939648677119553,0.029143186739241125,Watermarked +3483,0.5333333333333333,0.4394808743169399,-2.7199087833206432,0.07255497251636857,Watermarked +3484,0.6,0.32766393442622954,-7.616943405099432,0.004696954329420175,Watermarked +3485,0.6,0.41454918032786886,-4.203244682830465,0.024581754588030094,Watermarked +3486,0.6065573770491803,0.36666666666666664,-8.088416484282563,0.003948972398783288,Watermarked +3487,0.5666666666666667,0.35601092896174863,-2.719036894075117,0.07260835021632396,Watermarked +3488,0.65,0.3983606557377049,-10.650424052370854,0.001769113584875861,Watermarked +3489,0.55,0.43558743169398906,-2.9888817184112546,0.058182556819309464,Watermarked +3490,0.6,0.3937841530054645,-4.000584919681885,0.027997738989114628,Watermarked +3491,0.639344262295082,0.3958333333333333,-5.43407451169032,0.012232865280609554,Watermarked +3492,0.7213114754098361,0.5166666666666667,-5.7882292798767665,0.010257246511227655,Watermarked +3493,0.55,0.3567622950819672,-20.143694223280193,0.00026743218087057065,Watermarked +3494,0.48333333333333334,0.3688524590163934,-2.2635535190344163,0.10856902810055359,Watermarked +3495,0.6229508196721312,0.3791666666666667,-5.261267526539542,0.013378770206243135,Watermarked +3496,0.5833333333333334,0.4192622950819672,-4.998024144164243,0.01540912351567112,Watermarked +3497,0.6,0.4185792349726776,-4.200526985325906,0.024623895827933763,Watermarked +3498,0.6,0.3777322404371585,-3.6747652949491756,0.034884091484984066,Watermarked +3499,0.6885245901639344,0.3916666666666667,-27.593419053471173,0.00010447312619049549,Watermarked +3500,0.6229508196721312,0.41666666666666663,-5.199400639874496,0.013822982018819301,Watermarked +3501,0.6333333333333333,0.46427595628415297,-4.318179445331662,0.022881712741708818,Watermarked +3502,0.48333333333333334,0.3237704918032787,-4.892471154105429,0.016335808762118268,Watermarked +3503,0.5245901639344263,0.39166666666666666,-3.8686420190221638,0.030555105638903693,Watermarked +3504,0.5666666666666667,0.37766393442622953,-3.0183385183635814,0.05683401011881442,Watermarked +3505,0.5666666666666667,0.4062158469945355,-3.358719055760332,0.043774850277544984,Watermarked +3506,0.5573770491803278,0.43333333333333335,-6.89052992289735,0.006262188755552419,Watermarked +3507,0.6065573770491803,0.42083333333333334,-8.914754098360653,0.00297722406530442,Watermarked +3508,0.5245901639344263,0.3416666666666666,-7.603986987982712,0.004720052083609356,Watermarked +3509,0.45,0.3691939890710383,-1.986395773541501,0.14117716016377596,Watermarked +3510,0.55,0.360724043715847,-4.946082903302941,0.01585632915848264,Watermarked +3511,0.5737704918032787,0.36666666666666664,-4.588689352245944,0.019441198543417786,Watermarked +3512,0.6885245901639344,0.3375,-12.748930752673918,0.0010411491892625435,Watermarked +3513,0.7377049180327869,0.32499999999999996,-12.2541580554229,0.0011703222583523416,Watermarked +3514,0.6666666666666666,0.3778005464480874,-4.797285861708767,0.017234618038032837,Watermarked +3515,0.7666666666666667,0.34405737704918027,-14.172745497720056,0.0007609909456212724,Watermarked +3516,0.6557377049180327,0.37083333333333335,-5.3888664006939155,0.012519846650421048,Watermarked +3517,0.6721311475409836,0.4041666666666667,-8.518263011092074,0.003398441269028304,Watermarked +3518,0.6721311475409836,0.3458333333333333,-9.713343067748339,0.002317591752507357,Watermarked +3519,0.6229508196721312,0.30833333333333335,-6.506748841466283,0.007372760445320987,Watermarked +3520,0.5245901639344263,0.42083333333333334,-2.8626986790948408,0.06443431083077424,Watermarked +3521,0.6166666666666667,0.4065573770491804,-6.594241817524086,0.007098081515958769,Watermarked +3522,0.5666666666666667,0.34419398907103826,-9.378588970336171,0.0025678094646998427,Watermarked +3523,0.5833333333333334,0.3485655737704918,-3.9566860100053627,0.028816950912038987,Watermarked +3524,0.6166666666666667,0.3726775956284153,-3.463270042711585,0.040543787476660896,Watermarked +3525,0.5166666666666667,0.43586065573770494,-4.117687550148067,0.02595439887765405,Watermarked +3526,0.6065573770491803,0.42083333333333334,-2.7244712370027178,0.07227644833071428,Watermarked +3527,0.65,0.37315573770491806,-7.357526587712183,0.005189458410858861,Watermarked +3528,0.819672131147541,0.32083333333333336,-12.69043363555155,0.0010554010131434192,Watermarked +3529,0.5166666666666667,0.3372495446265938,-4.51546348372934,0.04570861042365545,Watermarked +3530,0.5573770491803278,0.3416666666666667,-3.2106696051862316,0.04893122743058373,Watermarked +3531,0.5245901639344263,0.3875,-2.707548854683142,0.07331622631866143,Watermarked +3532,0.5333333333333333,0.39398907103825137,-3.677608363112817,0.03481512208257183,Watermarked +3533,0.5666666666666667,0.4894808743169399,-2.350517220396054,0.1002587000234438,Watermarked +3534,0.5333333333333333,0.4025273224043716,-3.2715037607881903,0.04672400543382343,Watermarked +3535,0.5833333333333334,0.36441256830601093,-3.6162952233041508,0.036342061007907864,Watermarked +3536,0.6666666666666666,0.4438524590163935,-17.78543820658479,0.0003875755523465665,Watermarked +3537,0.639344262295082,0.3333333333333333,-5.158898896581134,0.014124257169801814,Watermarked +3538,0.6166666666666667,0.49760928961748635,-3.887955865194524,0.030162518753766708,Watermarked +3539,0.6333333333333333,0.4562841530054645,-9.590620075287022,0.0024054083955540014,Watermarked +3540,0.6229508196721312,0.4083333333333333,-3.897367233401582,0.029973563805789114,Watermarked +3541,0.6229508196721312,0.4,-6.306001458450521,0.008057963024016634,Watermarked +3542,0.5166666666666667,0.40628415300546444,-2.0963572160561217,0.1270050338262828,Watermarked +3543,0.5245901639344263,0.40416666666666673,-2.1561984129969214,0.12000461051054315,Watermarked +3544,0.5333333333333333,0.4229508196721311,-4.136976592067392,0.025636466240534804,Watermarked +3545,0.5833333333333334,0.42691256830601093,-3.084110518603815,0.053961847840379426,Watermarked +3546,0.5573770491803278,0.35833333333333334,-4.84204603440025,0.016804152704825513,Watermarked +3547,0.6,0.4023907103825136,-6.903695753068684,0.006228128127498301,Watermarked +3548,0.6885245901639344,0.37083333333333335,-22.322514291558807,0.00019683968706694644,Watermarked +3549,0.48333333333333334,0.32586520947176684,-3.2146877218511154,0.08465882990859916,Watermarked +3550,0.65,0.38995901639344266,-19.23389711083783,0.0003069437053761336,Watermarked +3551,0.6166666666666667,0.44815573770491807,-12.270797400644883,0.0011656421347428028,Watermarked +3552,0.6065573770491803,0.37083333333333335,-3.3590074850087963,0.043765495024786334,Watermarked +3553,0.6229508196721312,0.3208333333333333,-8.192450020709705,0.003805499272772318,Watermarked +3554,0.5833333333333334,0.5062841530054645,-5.14902114352389,0.01419903025134191,Watermarked +3555,0.6166666666666667,0.38545081967213113,-6.522157996146436,0.007323382596367344,Watermarked +3556,0.5245901639344263,0.3625,-5.036188941644216,0.015090956088189262,Watermarked +3557,0.5573770491803278,0.38333333333333336,-4.5941415869804265,0.019378995099920374,Watermarked +3558,0.5166666666666667,0.46045081967213114,-2.451392373266136,0.09156587040440396,Watermarked +3559,0.5409836065573771,0.4041666666666667,-3.3807925592909616,0.04306616942393178,Watermarked +3560,0.5409836065573771,0.49583333333333335,-2.6281319368691185,0.07845092410121779,Watermarked +3561,0.7166666666666667,0.38551912568306007,-7.359999679334292,0.0051844490931802715,Watermarked +3562,0.7166666666666667,0.31905737704918036,-9.174911614798257,0.002737770634718071,Watermarked +3563,0.6065573770491803,0.4083333333333333,-3.0623788546160213,0.054890208233557496,Watermarked +3564,0.5333333333333333,0.43041894353369764,-1.6922369290882864,0.23267507692719033,Watermarked +3565,0.6666666666666666,0.36489071038251364,-7.458228376445201,0.004990536830131611,Watermarked +3566,0.5166666666666667,0.39842896174863385,-14.529811045035604,0.000706861443434645,Watermarked +3567,0.5666666666666667,0.3861338797814208,-3.095250278326531,0.05349362654863495,Watermarked +3568,0.6885245901639344,0.3375,-8.553875315773226,0.003357503807386855,Watermarked +3569,0.5333333333333333,0.4355191256830601,-7.054729326958077,0.0058542974670971835,Watermarked +3570,0.6557377049180327,0.39583333333333326,-4.287445285495584,0.023321028120091257,Watermarked +3571,0.6333333333333333,0.38989071038251366,-11.606555225890991,0.0013736462375547754,Watermarked +3572,0.6166666666666667,0.40211748633879785,-3.9527199474578647,0.028892473194063042,Watermarked +3573,0.5081967213114754,0.3875,-2.3837761964250683,0.09728573796120057,Watermarked +3574,0.6557377049180327,0.3958333333333333,-5.180131981040665,0.013965256620241263,Watermarked +3575,0.639344262295082,0.48750000000000004,-5.206088992974237,0.01377403946898648,Watermarked +3576,0.45,0.3984289617486339,-3.252508058107846,0.047399438327312575,Watermarked +3577,0.6229508196721312,0.45833333333333326,-3.755595353518015,0.032989461923480595,Watermarked +3578,0.6721311475409836,0.43333333333333335,-5.352084912420385,0.012759859239265056,Watermarked +3579,0.55,0.3859289617486339,-6.327570523296936,0.007980431881706069,Watermarked +3580,0.6885245901639344,0.3583333333333333,-8.261956373376224,0.0037134622752403278,Watermarked +3581,0.6065573770491803,0.39583333333333337,-14.806483893669949,0.0006683885463905405,Watermarked +3582,0.5409836065573771,0.375,-6.0055128469177514,0.009248776088665693,Watermarked +3583,0.7333333333333333,0.3855874316939891,-13.52760363997253,0.0008736345306870901,Watermarked +3584,0.5901639344262295,0.375,-5.923439030264504,0.009613733901387398,Watermarked +3585,0.7704918032786885,0.32499999999999996,-15.651197399435246,0.0005668681892135681,Watermarked +3586,0.7704918032786885,0.3041666666666667,-10.064072122074942,0.0020889346595399047,Watermarked +3587,0.5409836065573771,0.3625,-4.290763799723821,0.02327307093304941,Watermarked +3588,0.6721311475409836,0.37916666666666665,-7.686877814143884,0.004574822356362937,Watermarked +3589,0.5166666666666667,0.24057377049180326,-10.793811013450267,0.0017009322854444507,Watermarked +3590,0.55,0.40635245901639344,-3.8261690084495177,0.03144179376080752,Watermarked +3591,0.55,0.4396857923497268,-6.979319263082819,0.006037138823969032,Watermarked +3592,0.639344262295082,0.35833333333333334,-5.646420206490955,0.010994199496502145,Watermarked +3593,0.5,0.35724043715847,-2.2370139019506667,0.11126942224811241,Watermarked +3594,0.5666666666666667,0.4478825136612022,-5.2484139227280515,0.01346950981253263,Watermarked +3595,0.5245901639344263,0.4541666666666667,-3.811215023955327,0.03176182755813695,Watermarked +3596,0.6166666666666667,0.3770491803278689,-5.292311191396577,0.013162886761926772,Watermarked +3597,0.8333333333333334,0.4108606557377049,-28.554907520499913,9.430057361661532e-05,Watermarked +3598,0.5666666666666667,0.4064207650273224,-5.716648520223962,0.01062086627428458,Watermarked +3599,0.5666666666666667,0.4354508196721312,-6.567850831861847,0.007179495661119332,Watermarked +3600,0.6557377049180327,0.4625,-4.525936567821964,0.02017613595003384,Watermarked +3601,0.5666666666666667,0.2616120218579235,-5.789061661601357,0.010253114813387197,Watermarked +3602,0.6065573770491803,0.4208333333333333,-4.46482463507427,0.020926850914564546,Watermarked +3603,0.6166666666666667,0.373292349726776,-5.837995555994575,0.010014006051520334,Watermarked +3604,0.6229508196721312,0.32083333333333336,-5.409452966923079,0.012388083734135366,Watermarked +3605,0.7213114754098361,0.37083333333333335,-13.136517577876024,0.0009528915088720033,Watermarked +3606,0.45,0.38565573770491807,-1.9388846252338177,0.1478731519159202,Watermarked +3607,0.65,0.33975409836065573,-7.109185568026944,0.005726775618868671,Watermarked +3608,0.6166666666666667,0.4028005464480875,-4.245647176969608,0.02393615072729287,Watermarked +3609,0.5333333333333333,0.31509562841530053,-8.1350398352272,0.003883805394600376,Watermarked +3610,0.6065573770491803,0.275,-8.547594094887065,0.0033646767112774094,Watermarked +3611,0.5166666666666667,0.37739071038251365,-4.570466183924919,0.019650992181636566,Watermarked +3612,0.7049180327868853,0.38749999999999996,-10.334980525866824,0.001932389182639011,Watermarked +3613,0.65,0.46461748633879785,-6.40281942286815,0.007717499991952441,Watermarked +3614,0.5245901639344263,0.39166666666666666,-4.3147109239249755,0.022930753286261416,Watermarked +3615,0.5166666666666667,0.3690573770491803,-3.785362525900684,0.0323250246243217,Watermarked +3616,0.5666666666666667,0.4151639344262295,-3.7566753428966173,0.03296505095792107,Watermarked +3617,0.5409836065573771,0.35416666666666663,-17.816045884784717,0.0003855963684750506,Watermarked +3618,0.5333333333333333,0.35239071038251363,-4.290395689342995,0.023278384354672565,Watermarked +3619,0.8032786885245902,0.4291666666666667,-17.0700379678159,0.00043795305550041073,Watermarked +3620,0.7213114754098361,0.4,-27.264182776241938,0.00010829140307033679,Watermarked +3621,0.5245901639344263,0.4166666666666667,-3.172289994751985,0.05039180926567533,Watermarked +3622,0.45,0.3867030965391621,-18.78378378378386,0.0028222292471545055,Watermarked +3623,0.5573770491803278,0.35833333333333334,-6.460980283821876,0.007522028995284054,Watermarked +3624,0.5573770491803278,0.4041666666666667,-8.291518233493496,0.0036752095251250446,Watermarked +3625,0.55,0.38524590163934425,-2.953558943086598,0.059853030783475765,Watermarked +3626,0.6166666666666667,0.4278005464480874,-3.220215216567121,0.048576294366246366,Watermarked +3627,0.5245901639344263,0.35416666666666663,-7.776108221599537,0.00442500536007881,Watermarked +3628,0.6229508196721312,0.3375,-6.2978017747795265,0.00808769678640319,Watermarked +3629,0.5,0.39392076502732243,-2.8228292898095364,0.06658256000513207,Watermarked +3630,0.6333333333333333,0.3321038251366121,-9.748782925528516,0.002293024514092687,Watermarked +3631,0.5,0.3729508196721311,-3.16062146897757,0.05084675449389502,Watermarked +3632,0.6666666666666666,0.31974043715846995,-16.761645970184002,0.0004623636950420215,Watermarked +3633,0.5666666666666667,0.4396174863387978,-7.124258314479804,0.0056921272252719556,Watermarked +3634,0.6333333333333333,0.46072404371584696,-5.791146439440572,0.010242776156530948,Watermarked +3635,0.5409836065573771,0.4125,-7.478844437595781,0.00495105882203386,Watermarked +3636,0.6666666666666666,0.4194672131147541,-3.705356086132572,0.0341510693846338,Watermarked +3637,0.6065573770491803,0.2583333333333333,-11.303389666068615,0.0014850505321337656,Watermarked +3638,0.4262295081967213,0.37499999999999994,-2.582483761527067,0.0816038163981699,Watermarked +3639,0.6333333333333333,0.3319672131147541,-16.69265269416492,0.00046807129563190304,Watermarked +3640,0.6,0.42718579234972676,-4.989229131610327,0.01548368101755126,Watermarked +3641,0.7049180327868853,0.37916666666666665,-11.48552138205233,0.0014167460006659608,Watermarked +3642,0.5,0.3734289617486339,-7.373424834781318,0.0051573670428228195,Watermarked +3643,0.8,0.3737704918032787,-11.276355501471617,0.0014955591121438806,Watermarked +3644,0.6885245901639344,0.35833333333333334,-6.697501094028932,0.006790960911751886,Watermarked +3645,0.7377049180327869,0.37916666666666665,-18.208323627746466,0.0003613808590242596,Watermarked +3646,0.7213114754098361,0.47500000000000003,-5.911475409836067,0.009668500550400335,Watermarked +3647,0.5573770491803278,0.35,-3.567189130512097,0.03762717652367037,Watermarked +3648,0.5,0.4062841530054645,-1.9500758855993163,0.1462629282557643,Watermarked +3649,0.7049180327868853,0.3291666666666666,-35.834028669908896,4.779336400647669e-05,Watermarked +3650,0.55,0.36475409836065575,-5.115536577871306,0.014456377843406167,Watermarked +3651,0.5166666666666667,0.3778005464480874,-2.8771732821175253,0.06367584045734935,Watermarked +3652,0.5166666666666667,0.39064207650273225,-2.5781209852352887,0.08191326217312117,Watermarked +3653,0.6333333333333333,0.398155737704918,-6.811947916107325,0.006470670821303707,Watermarked +3654,0.5573770491803278,0.35416666666666663,-3.722323789331516,0.03375297717263195,Watermarked +3655,0.6333333333333333,0.40225409836065573,-7.458643842395928,0.004989737155548977,Watermarked +3656,0.5409836065573771,0.39583333333333337,-4.726024019252636,0.017950011968011444,Watermarked +3657,0.5573770491803278,0.2833333333333333,-23.253380378036304,0.00017423225248505516,Watermarked +3658,0.6229508196721312,0.4375,-3.696204554145968,0.034368276506610014,Watermarked +3659,0.48333333333333334,0.41976320582877963,-5.126120909780012,0.03601274479900024,Watermarked +3660,0.4918032786885246,0.31666666666666665,-3.5694569453443568,0.03756656146901733,Watermarked +3661,0.7049180327868853,0.31666666666666665,-19.02035477866064,0.000317329217166385,Watermarked +3662,0.6833333333333333,0.2987704918032787,-9.287563234189735,0.002641987913158529,Watermarked +3663,0.5901639344262295,0.35,-20.378585595507392,0.00025834295408300217,Watermarked +3664,0.5666666666666667,0.448292349726776,-3.5599428053606994,0.03782169758880093,Watermarked +3665,0.6166666666666667,0.4021857923497268,-8.675666340422445,0.0032223344164688237,Watermarked +3666,0.6885245901639344,0.3958333333333333,-12.228241897871557,0.0011776614876831963,Watermarked +3667,0.6333333333333333,0.4060109289617486,-4.451765940749828,0.0210919401610888,Watermarked +3668,0.6,0.3400956284153005,-16.126211408017976,0.0005186735594686228,Watermarked +3669,0.6721311475409836,0.3583333333333334,-21.740550300468346,0.00021299095231525587,Watermarked +3670,0.6,0.4187158469945355,-5.0760762819997485,0.014767517268100116,Watermarked +3671,0.5833333333333334,0.4314890710382514,-4.691101983797846,0.018314699259855377,Watermarked +3672,0.6166666666666667,0.4105191256830601,-6.871714846368723,0.006311291308420261,Watermarked +3673,0.5666666666666667,0.4023907103825136,-3.1785087830863152,0.050151443196549406,Watermarked +3674,0.5666666666666667,0.38989071038251366,-8.909952627909695,0.002981899212734121,Watermarked +3675,0.6666666666666666,0.3488387978142076,-10.158428205179094,0.0020325642199010164,Watermarked +3676,0.5,0.39016393442622954,-4.155570602285484,0.025334728700013992,Watermarked +3677,0.6,0.426844262295082,-3.4665463985771736,0.04044752053565438,Watermarked +3678,0.65,0.42295081967213116,-5.864413389759132,0.009887952398219825,Watermarked +3679,0.6,0.4019808743169399,-3.3292886989365154,0.044742920567822704,Watermarked +3680,0.639344262295082,0.39999999999999997,-9.082476164942625,0.0028198267013386054,Watermarked +3681,0.5081967213114754,0.3541666666666667,-5.142940376750061,0.014245317657868042,Watermarked +3682,0.7704918032786885,0.3791666666666667,-10.107871974839727,0.0020625133025401216,Watermarked +3683,0.7704918032786885,0.41666666666666663,-15.679030900154354,0.0005638836832955063,Watermarked +3684,0.6721311475409836,0.41666666666666663,-12.515152510941379,0.0010996877982163017,Watermarked +3685,0.5333333333333333,0.4605191256830601,-4.084201256082031,0.02651851352137615,Watermarked +3686,0.55,0.4396857923497267,-2.8054956411927168,0.06754435460228396,Watermarked +3687,0.6721311475409836,0.3416666666666667,-10.239067447514469,0.001985976142870652,Watermarked +3688,0.5901639344262295,0.42083333333333334,-5.382812644531547,0.012558941549593422,Watermarked +3689,0.5,0.42711748633879776,-1.4944177904953027,0.2319280697354033,Watermarked +3690,0.55,0.40232240437158473,-6.785424665478694,0.006543098179519793,Watermarked +3691,0.5333333333333333,0.3818989071038251,-3.1559440965062726,0.05103057602991868,Watermarked +3692,0.5737704918032787,0.375,-3.141020234264478,0.051622720218920067,Watermarked +3693,0.5333333333333333,0.33706739526411655,-12.955859792395307,0.005904832207349002,Watermarked +3694,0.7704918032786885,0.32916666666666666,-8.41751169737434,0.0035178920060083122,Watermarked +3695,0.5409836065573771,0.4541666666666666,-3.061039563134522,0.0549480767673967,Watermarked +3696,0.55,0.39822404371584696,-3.1169399358461005,0.052596564456531714,Watermarked +3697,0.6229508196721312,0.425,-4.093911885056293,0.026353313515894105,Watermarked +3698,0.6,0.41072404371584703,-16.306113587728134,0.0005018457959471869,Watermarked +3699,0.6229508196721312,0.46249999999999997,-3.08642244690056,0.05386425088654483,Watermarked +3700,0.5333333333333333,0.43155737704918035,-7.41608714028745,0.00507253495755017,Watermarked +3701,0.639344262295082,0.42916666666666664,-3.7914997183027954,0.03219017406449965,Watermarked +3702,0.55,0.4023224043715847,-5.3872290083001095,0.012530405150481858,Watermarked +3703,0.5833333333333334,0.3773224043715847,-4.640651750707308,0.018858726540020358,Watermarked +3704,0.48333333333333334,0.3860655737704918,-3.4475208334165512,0.04101062763956584,Watermarked +3705,0.6229508196721312,0.4333333333333334,-5.465357642986741,0.01203929791125337,Watermarked +3706,0.5081967213114754,0.3458333333333333,-3.3291936924471166,0.04474608942509305,Watermarked +3707,0.48333333333333334,0.37315573770491806,-2.5737901679797495,0.0822218718502907,Watermarked +3708,0.5573770491803278,0.41666666666666663,-9.248430069349528,0.0026747507004180183,Watermarked +3709,0.6333333333333333,0.3398224043715847,-7.915878353618565,0.004203089684479934,Watermarked +3710,0.6333333333333333,0.419603825136612,-4.140759720616552,0.025574700755914715,Watermarked +3711,0.6666666666666666,0.4316939890710383,-11.006210975743855,0.0016061976208647793,Watermarked +3712,0.5,0.38627049180327866,-2.911570443803633,0.06191769354930334,Watermarked +3713,0.5833333333333334,0.46427595628415297,-1.8536473961974427,0.16084088720625275,Watermarked +3714,0.5409836065573771,0.4125,-2.7434685305994506,0.07113088858607176,Watermarked +3715,0.55,0.3528688524590164,-4.518525749449755,0.02026529222371772,Watermarked +3716,0.5666666666666667,0.4687158469945355,-6.390672157378807,0.007759165768176083,Watermarked +3717,0.6333333333333333,0.39788251366120214,-5.975152199128738,0.009381627777611113,Watermarked +3718,0.5901639344262295,0.44583333333333336,-4.1303709900535255,0.025744775027266065,Watermarked +3719,0.6,0.32766393442622954,-15.565235673998512,0.0005762193133083165,Watermarked +3720,0.5737704918032787,0.39166666666666666,-6.397744904460955,0.007734870011183786,Watermarked +3721,0.5573770491803278,0.41666666666666663,-2.647817826363915,0.0771377198138251,Watermarked +3722,0.5833333333333334,0.43128415300546447,-3.57266399939355,0.03748105460475532,Watermarked +3723,0.5737704918032787,0.4625,-2.7495312426784695,0.07077005262877357,Watermarked +3724,0.639344262295082,0.3291666666666666,-16.78635055652876,0.00046034248285934797,Watermarked +3725,0.5409836065573771,0.37083333333333335,-3.985191937237371,0.02828153983006206,Watermarked +3726,0.6166666666666667,0.3559426229508197,-3.590780809031045,0.037002659109130365,Watermarked +3727,0.8360655737704918,0.30833333333333335,-25.163943419716063,0.00013761668634114897,Watermarked +3728,0.8360655737704918,0.30833333333333335,-14.528409507245575,0.0007070637377049956,Watermarked +3729,0.6333333333333333,0.4310109289617486,-3.9857409240546327,0.028271354653200294,Watermarked +3730,0.5245901639344263,0.2958333333333334,-5.2910842726589395,0.013171332285404133,Watermarked +3731,0.6166666666666667,0.3568306010928962,-24.813855969761136,0.00014350088541621058,Watermarked +3732,0.7704918032786885,0.45,-21.064870818149505,0.00023403567730180875,Watermarked +3733,0.55,0.38135245901639336,-5.256718568825,0.013410791921047589,Watermarked +3734,0.4666666666666667,0.39009562841530054,-2.671727253283191,0.07557911199765509,Watermarked +3735,0.5666666666666667,0.4108606557377049,-3.0138201768645576,0.05703829450534057,Watermarked +3736,0.6833333333333333,0.31509562841530053,-12.567253014018696,0.0010862702962580325,Watermarked +3737,0.5409836065573771,0.48333333333333334,-8.472825175856576,0.003451637475653648,Watermarked +3738,0.7704918032786885,0.3583333333333333,-12.770263120773526,0.001036015557970406,Watermarked +3739,0.4918032786885246,0.3708333333333333,-2.3891724073960368,0.09681354824211097,Watermarked +3740,0.7,0.4483606557377049,-12.885005111050303,0.0010089720709264307,Watermarked +3741,0.6885245901639344,0.40416666666666673,-5.830640867335859,0.010049473664175738,Watermarked +3742,0.5737704918032787,0.37083333333333335,-10.982657455064276,0.0016163522474045274,Watermarked +3743,0.5666666666666667,0.38954918032786884,-4.0760180019956564,0.02665876696412885,Watermarked +3744,0.5166666666666667,0.3447404371584699,-3.192419176892379,0.0496190039928642,Watermarked +3745,0.5,0.4438524590163935,-1.784232222780322,0.17238357843904029,Watermarked +3746,0.5,0.4143897996357013,-8.657804801663772,0.013079710332824616,Watermarked +3747,0.6065573770491803,0.3875,-5.491151239179569,0.011882700028816374,Watermarked +3748,0.6333333333333333,0.46885245901639344,-3.6820317950825556,0.03470816139558017,Watermarked +3749,0.6229508196721312,0.47500000000000003,-3.5044017326627706,0.03935606956265976,Watermarked +3750,0.7704918032786885,0.3458333333333333,-15.423254035682769,0.0005921188778650158,Watermarked +3751,0.75,0.42745901639344264,-24.49428445453386,0.00014916835502607742,Watermarked +3752,0.7540983606557377,0.4041666666666667,-11.683939490062333,0.0013469985474943292,Watermarked +3753,0.5737704918032787,0.4125,-3.7772132331663713,0.03250520782748611,Watermarked +3754,0.6885245901639344,0.3,-10.255681847231036,0.0019765537724212886,Watermarked +3755,0.48333333333333334,0.3360655737704918,-10.610181401233849,0.00178889887952055,Watermarked +3756,0.8360655737704918,0.3875,-18.026271996123526,0.0003723594365652728,Watermarked +3757,0.6833333333333333,0.3524590163934427,-9.375725828304732,0.002570100386249655,Watermarked +3758,0.6885245901639344,0.45833333333333337,-7.472032792203066,0.004964056821106828,Watermarked +3759,0.5,0.4308743169398907,-4.435866726830125,0.04724822608025257,Watermarked +3760,0.6557377049180327,0.4208333333333334,-5.724222068679529,0.010581602969978048,Watermarked +3761,0.6166666666666667,0.4349726775956284,-3.1471966691041255,0.05137660775019963,Watermarked +3762,0.55,0.4689207650273224,-2.927463218319362,0.0611258808814014,Watermarked +3763,0.4,0.2701502732240437,-3.40514995545028,0.042300988302641715,Watermarked +3764,0.5833333333333334,0.4566256830601093,-3.4235368770297985,0.04173478390661575,Watermarked +3765,0.55,0.4398224043715847,-10.558130916542433,0.0018149271537199203,Watermarked +3766,0.6557377049180327,0.45416666666666666,-5.7684452397766,0.010356094063164631,Watermarked +3767,0.5166666666666667,0.41454918032786886,-2.3715832376714867,0.09836303411399833,Watermarked +3768,0.55,0.3109289617486338,-6.228218140785886,0.008345917876290724,Watermarked +3769,0.5573770491803278,0.3916666666666667,-5.9956272290627375,0.00929176090610574,Watermarked +3770,0.6666666666666666,0.42745901639344264,-26.610860299950033,0.00011643687772756474,Watermarked +3771,0.5081967213114754,0.35000000000000003,-3.224205259523217,0.048428898137238975,Watermarked +3772,0.7213114754098361,0.3916666666666666,-8.7724963613773,0.0031199616023164897,Watermarked +3773,0.55,0.33128415300546443,-3.6830493742159134,0.03468361534744764,Watermarked +3774,0.5901639344262295,0.4125,-4.161173917605138,0.025244699113682725,Watermarked +3775,0.6065573770491803,0.49583333333333335,-2.859986118208274,0.0645777020694818,Watermarked +3776,0.5666666666666667,0.435724043715847,-3.1832159126292936,0.04997046721617775,Watermarked +3777,0.5901639344262295,0.4166666666666667,-8.499595555559111,0.003420163998746116,Watermarked +3778,0.5666666666666667,0.46038251366120214,-2.3871021119027693,0.09699437660783286,Watermarked +3779,0.4666666666666667,0.3898224043715847,-1.856072621295239,0.16045407134930564,Watermarked +3780,0.5245901639344263,0.3833333333333333,-3.203403812225747,0.049203590003784436,Watermarked +3781,0.6557377049180327,0.4000000000000001,-37.585613102377934,4.1428594619016965e-05,Watermarked +3782,0.6557377049180327,0.3625,-9.790972522123287,0.002264226155807123,Watermarked +3783,0.5666666666666667,0.3983606557377049,-5.666139775569049,0.010887658149858946,Watermarked +3784,0.5666666666666667,0.37698087431693994,-3.066930962984633,0.054694091557813726,Watermarked +3785,0.5737704918032787,0.4291666666666667,-3.794152560630122,0.03213210729307197,Watermarked +3786,0.5901639344262295,0.4375,-4.005635792635015,0.027905415031102557,Watermarked +3787,0.6166666666666667,0.4398907103825137,-15.578682000811366,0.0005747431214231546,Watermarked +3788,0.5333333333333333,0.4480191256830601,-6.166892456957159,0.008582542762464446,Watermarked +3789,0.6166666666666667,0.44760928961748636,-3.4952014484831047,0.039617848196352085,Watermarked +3790,0.5737704918032787,0.3541666666666667,-3.9615442295558925,0.028724785655088337,Watermarked +3791,0.8032786885245902,0.29583333333333334,-12.539145243620842,0.0010934817370295948,Watermarked +3792,0.6557377049180327,0.45416666666666666,-7.1071028723917875,0.0057315850409424066,Watermarked +3793,0.47540983606557374,0.42083333333333334,-6.840395759439469,0.006394156517694729,Watermarked +3794,0.6557377049180327,0.4083333333333334,-4.492763882679988,0.020579223849711902,Watermarked +3795,0.5333333333333333,0.41905737704918034,-6.292690992312831,0.008106302500641403,Watermarked +3796,0.7,0.4064890710382514,-7.456642804764441,0.004993590238372753,Watermarked +3797,0.5573770491803278,0.4083333333333334,-2.8639314065795065,0.06436927783811348,Watermarked +3798,0.6229508196721312,0.4416666666666667,-4.673530022214723,0.018501844031635108,Watermarked +3799,0.6065573770491803,0.3916666666666667,-4.31785074614014,0.02288635431540596,Watermarked +3800,0.7704918032786885,0.3125,-21.98360655737705,0.0002060384328133572,Watermarked +3801,0.6065573770491803,0.38333333333333336,-4.785407410045699,0.017351257215718095,Watermarked +3802,0.45901639344262296,0.35,-3.0834492425511746,0.05398980423292988,Watermarked +3803,0.5573770491803278,0.39166666666666666,-3.7805287666533993,0.032431745470329315,Watermarked +3804,0.6065573770491803,0.39166666666666666,-5.539911623921449,0.011593870922493347,Watermarked +3805,0.5833333333333334,0.410724043715847,-11.448148657407378,0.0014304164055433837,Watermarked +3806,0.6065573770491803,0.4291666666666667,-3.064527056875999,0.05479754804131916,Watermarked +3807,0.5666666666666667,0.4691256830601093,-4.840779946644281,0.016816135457307743,Watermarked +3808,0.5666666666666667,0.45669398907103825,-4.048874395684384,0.02713088105493987,Watermarked +3809,0.5573770491803278,0.39583333333333337,-2.771678715670956,0.0694710760166772,Watermarked +3810,0.5166666666666667,0.44836065573770495,-3.137402547907732,0.051767564979331604,Watermarked +3811,0.5409836065573771,0.35833333333333334,-9.207415437764503,0.0027096685823184735,Watermarked +3812,0.6065573770491803,0.37916666666666665,-6.7690431384831955,0.006588365191149356,Watermarked +3813,0.5833333333333334,0.43995901639344265,-6.955531695206198,0.0060963729996066736,Watermarked +3814,0.5573770491803278,0.37916666666666665,-4.17397260708643,0.02504060302241229,Watermarked +3815,0.7213114754098361,0.3875,-13.414708580029796,0.0008955843183091936,Watermarked +3816,0.6166666666666667,0.5266393442622951,-2.0100838668302545,0.13797226511673932,Watermarked +3817,0.6166666666666667,0.3726775956284153,-3.954974925804998,0.028849502427145693,Watermarked +3818,0.7049180327868853,0.44583333333333336,-6.313455744839042,0.00803105673664604,Watermarked +3819,0.6166666666666667,0.44815573770491807,-4.067460467766922,0.02680645933365517,Watermarked +3820,0.5737704918032787,0.475,-3.0602917588065157,0.054980421525545886,Watermarked +3821,0.4666666666666667,0.3775273224043716,-9.402775448335925,0.0025485643327779633,Watermarked +3822,0.6166666666666667,0.36489071038251364,-8.409157084603672,0.003528044949249387,Watermarked +3823,0.5333333333333333,0.42725409836065575,-3.221457895636802,0.04853032779203252,Watermarked +3824,0.6229508196721312,0.4625,-3.0100455180506156,0.05720966272875764,Watermarked +3825,0.55,0.385792349726776,-4.6686976061393075,0.018553745232426106,Watermarked +3826,0.6,0.37732240437158465,-5.919359446594679,0.009632363464382891,Watermarked +3827,0.5573770491803278,0.45,-2.3790961945021833,0.0976975284567096,Watermarked +3828,0.6666666666666666,0.47274590163934427,-6.685134180648596,0.006826809830237584,Watermarked +3829,0.5409836065573771,0.42500000000000004,-1.7723627686294894,0.17445150254614536,Watermarked +3830,0.5833333333333334,0.4687158469945355,-2.42952422971014,0.09336973169699041,Watermarked +3831,0.5666666666666667,0.44781420765027324,-2.175312723682659,0.1178663975785963,Watermarked +3832,0.5666666666666667,0.42698087431693993,-2.926838266377036,0.06115677716147604,Watermarked +3833,0.6,0.4226775956284153,-3.0320096719785434,0.05622147844106881,Watermarked +3834,0.6885245901639344,0.42500000000000004,-8.165010792168419,0.0038426628271698176,Watermarked +3835,0.7166666666666667,0.40689890710382515,-13.281769925732059,0.00092238087463812,Watermarked +3836,0.7540983606557377,0.4541666666666667,-7.998178506375221,0.004079263296996052,Watermarked +3837,0.5901639344262295,0.4083333333333333,-9.166095697580067,0.002745459142872884,Watermarked +3838,0.4666666666666667,0.3903688524590164,-2.6475959716199546,0.07715236726431632,Watermarked +3839,0.5666666666666667,0.4352459016393443,-2.217999452996241,0.11325398002800678,Watermarked +3840,0.5409836065573771,0.4,-2.7939211613370594,0.06819623625179277,Watermarked +3841,0.5833333333333334,0.3810109289617486,-3.117346730699978,0.052579921647726034,Watermarked +3842,0.55,0.39419398907103825,-3.6055517374511754,0.036618371391318225,Watermarked +3843,0.5573770491803278,0.41250000000000003,-3.8013214837489318,0.03197586070428311,Watermarked +3844,0.5901639344262295,0.41666666666666663,-5.316864560968578,0.01299535017353051,Watermarked +3845,0.7049180327868853,0.37916666666666665,-11.831027585036374,0.001298210003952447,Watermarked +3846,0.639344262295082,0.4041666666666667,-8.063231850117097,0.003984775525196651,Watermarked +3847,0.5833333333333334,0.3689890710382514,-4.439726457213278,0.021245642150683707,Watermarked +3848,0.7166666666666667,0.3025273224043716,-11.897761281046622,0.001276847670564472,Watermarked +3849,0.6229508196721312,0.44166666666666665,-4.990732440102222,0.015470903850084424,Watermarked +3850,0.5833333333333334,0.3980874316939891,-3.5674165199732903,0.03762109314075504,Watermarked +3851,0.6333333333333333,0.40273224043715844,-4.589204955202506,0.019435305052459464,Watermarked +3852,0.6166666666666667,0.3774590163934426,-13.772608526452023,0.0008284008874866314,Watermarked +3853,0.55,0.39405737704918037,-4.944621149115616,0.015869157656486683,Watermarked +3854,0.4666666666666667,0.37315573770491806,-4.038348082595869,0.027316863729051996,Watermarked +3855,0.55,0.40259562841530055,-3.6156991795445577,0.0363573206604073,Watermarked +3856,0.6557377049180327,0.4,-10.42437348156729,0.0018841578114493912,Watermarked +3857,0.5333333333333333,0.4603142076502732,-2.0946414252764485,0.1272128293571138,Watermarked +3858,0.5833333333333334,0.41905737704918034,-3.9625676262993768,0.02870541916232176,Watermarked +3859,0.7377049180327869,0.29166666666666663,-14.478449297651643,0.0007143254209499869,Watermarked +3860,0.5737704918032787,0.38749999999999996,-10.080682221460885,0.0020788624411748344,Watermarked +3861,0.6721311475409836,0.37916666666666665,-13.36742611721058,0.0009049950751945303,Watermarked +3862,0.4918032786885246,0.45,-2.0479340472449516,0.13302782647013972,Watermarked +3863,0.639344262295082,0.42916666666666664,-4.986422813713202,0.015507569745157936,Watermarked +3864,0.5,0.441712204007286,-2.327272727272727,0.14541235094098032,Watermarked +3865,0.6833333333333333,0.45997267759562843,-3.7679579839771797,0.03271140841418524,Watermarked +3866,0.6166666666666667,0.4976775956284153,-5.300050196391964,0.013109777815587411,Watermarked +3867,0.8032786885245902,0.4041666666666667,-31.928961748633885,6.751260437384004e-05,Watermarked +3868,0.43333333333333335,0.3358606557377049,-2.42291804775351,0.09392318031625656,Watermarked +3869,0.6333333333333333,0.4194672131147541,-3.4472491733646837,0.041018740078086716,Watermarked +3870,0.5737704918032787,0.39583333333333337,-4.526712258059946,0.02016683338787022,Watermarked +3871,0.5833333333333334,0.48101092896174863,-3.9726503315641613,0.02851551151540454,Watermarked +3872,0.6333333333333333,0.4441256830601093,-9.737517564732535,0.00230079619793927,Watermarked +3873,0.7704918032786885,0.38749999999999996,-6.709762545561513,0.006755661879342611,Watermarked +3874,0.48333333333333334,0.3202185792349727,-8.067567567567563,0.015019105066650768,Watermarked +3875,0.5666666666666667,0.4229508196721311,-4.277954683338313,0.023458887518037853,Watermarked +3876,0.5,0.4062841530054645,-2.608245612118408,0.0798056461655953,Watermarked +3877,0.6166666666666667,0.423155737704918,-3.308126758133395,0.045455844114838176,Watermarked +3878,0.8032786885245902,0.2916666666666667,-13.189427229234468,0.0009416245667257401,Watermarked +3879,0.48333333333333334,0.40239071038251367,-2.8278186890378114,0.06630888074205549,Watermarked +3880,0.55,0.4270491803278688,-2.1944780188324806,0.11576816528411227,Watermarked +3881,0.5666666666666667,0.3773224043715847,-3.575689868461351,0.03740060612655586,Watermarked +3882,0.6065573770491803,0.43333333333333335,-6.573390283759843,0.007162305426399894,Watermarked +3883,0.7377049180327869,0.425,-7.134072833363263,0.005669714559524623,Watermarked +3884,0.5573770491803278,0.39999999999999997,-5.171937592179927,0.014026340881299086,Watermarked +3885,0.55,0.4687158469945355,-2.666321218691447,0.07592808657201537,Watermarked +3886,0.5833333333333334,0.4603142076502732,-3.6716000925111048,0.03496108120386053,Watermarked +3887,0.5333333333333333,0.41045081967213115,-1.9837288411299292,0.14154345106445665,Watermarked +3888,0.6,0.4400273224043716,-4.325688830675532,0.022776003224226602,Watermarked +3889,0.5166666666666667,0.42288251366120216,-2.3707547764032237,0.09843675721882074,Watermarked +3890,0.5901639344262295,0.42500000000000004,-3.0224716644609666,0.056647945467235325,Watermarked +3891,0.639344262295082,0.3791666666666667,-5.283658664721413,0.013222597948006223,Watermarked +3892,0.5666666666666667,0.36885245901639346,-2.7563407387163426,0.07036747746122053,Watermarked +3893,0.6721311475409836,0.32499999999999996,-16.552311734273484,0.0004799737975285094,Watermarked +3894,0.5666666666666667,0.43128415300546447,-5.492433465454693,0.011874985075137982,Watermarked +3895,0.6,0.4478142076502733,-4.942348751891569,0.01588912728706134,Watermarked +3896,0.6333333333333333,0.40225409836065573,-6.568502779421056,0.007177469697378324,Watermarked +3897,0.5333333333333333,0.3775273224043716,-10.30131962224801,0.0019509748009049809,Watermarked +3898,0.55,0.4235655737704918,-2.574541708593826,0.08216821527460087,Watermarked +3899,0.5333333333333333,0.45614754098360655,-2.6559601893971574,0.07660250967292412,Watermarked +3900,0.7049180327868853,0.32916666666666666,-11.674705169820832,0.0013501421719868342,Watermarked +3901,0.5666666666666667,0.4108606557377049,-6.418429391739877,0.007664387655280491,Watermarked +3902,0.7540983606557377,0.4208333333333334,-15.206265335945545,0.0006175648734416628,Watermarked +3903,0.5737704918032787,0.3625,-3.600412240200614,0.036751502378933966,Watermarked +3904,0.7,0.419603825136612,-5.8545065009896256,0.00993497840099364,Watermarked +3905,0.6333333333333333,0.4602459016393443,-5.345769495574332,0.012801672818638945,Watermarked +3906,0.5833333333333334,0.3818989071038251,-15.322633755583363,0.0006037415969847308,Watermarked +3907,0.5573770491803278,0.48750000000000004,-3.044989012722217,0.055647600875460466,Watermarked +3908,0.5901639344262295,0.45,-3.5859661472271527,0.03712903167578981,Watermarked +3909,0.5666666666666667,0.34822404371584703,-4.589291871691855,0.019434311800169517,Watermarked +3910,0.47540983606557374,0.35833333333333334,-3.1680031770414296,0.050558348943150866,Watermarked +3911,0.5901639344262295,0.3916666666666666,-13.752294116926654,0.0008320313667330868,Watermarked +3912,0.5833333333333334,0.36038251366120216,-4.510639994744203,0.02036072397240279,Watermarked +3913,0.4918032786885246,0.36666666666666664,-2.214046423571898,0.1136718934082652,Watermarked +3914,0.47540983606557374,0.44166666666666665,-2.3377953522924186,0.10142500753372925,Watermarked +3915,0.6833333333333333,0.3445355191256831,-8.421776890073554,0.0035127236241960158,Watermarked +3916,0.55,0.43196721311475406,-3.1667548890956354,0.05060697458442003,Watermarked +3917,0.5666666666666667,0.4644808743169399,-2.238658218669735,0.11109977799644323,Watermarked +3918,0.5,0.3859289617486339,-4.12442499010612,0.025842773831512598,Watermarked +3919,0.6166666666666667,0.3979508196721311,-6.410996076998129,0.00768961910808135,Watermarked +3920,0.5,0.38579234972677595,-3.247114142040317,0.04759348039328008,Watermarked +3921,0.7868852459016393,0.4,-32.828301710168866,6.212646695041301e-05,Watermarked +3922,0.5737704918032787,0.4,-9.652812843530201,0.002360364565844294,Watermarked +3923,0.7213114754098361,0.33749999999999997,-13.159250585480095,0.0009480287588880445,Watermarked +3924,0.6065573770491803,0.3958333333333333,-3.9531674853921452,0.02888393838786599,Watermarked +3925,0.6065573770491803,0.38750000000000007,-15.392023937439065,0.0005956941961595111,Watermarked +3926,0.5409836065573771,0.475,-2.1418349190706736,0.12164197076622749,Watermarked +3927,0.55,0.3730874316939891,-3.9173359194644175,0.029577655575744665,Watermarked +3928,0.6885245901639344,0.3583333333333334,-6.697501094028931,0.006790960911751886,Watermarked +3929,0.5833333333333334,0.4520491803278689,-3.381086105792825,0.04305684351738099,Watermarked +3930,0.45,0.34419398907103826,-5.4455564685402775,0.012161350131513373,Watermarked +3931,0.75,0.4474726775956285,-5.428996703057984,0.012264667663581029,Watermarked +3932,0.38333333333333336,0.3279371584699453,-4.7611795122867155,0.017592360487795486,Watermarked +3933,0.5,0.39009562841530054,-7.41768759066674,0.005069388475696104,Watermarked +3934,0.5081967213114754,0.42083333333333334,-1.9061102831594638,0.1527095539473995,Watermarked +3935,0.43333333333333335,0.38586065573770495,-2.611459754304495,0.07958474323908545,Watermarked +3936,0.7666666666666667,0.39009562841530054,-41.93436311911947,2.9845043324434687e-05,Watermarked +3937,0.6,0.40218579234972673,-4.703745486314339,0.01818156231348825,Watermarked +3938,0.5901639344262295,0.375,-5.5469553767808515,0.011552909711436831,Watermarked +3939,0.5409836065573771,0.43333333333333335,-2.4127247774526692,0.09478502280607064,Watermarked +3940,0.65,0.3896174863387978,-6.0971062067672115,0.008862646607214728,Watermarked +3941,0.5245901639344263,0.4375,-2.361603838210462,0.09925556793924133,Watermarked +3942,0.7377049180327869,0.4333333333333333,-7.354109955820557,0.0051963894030398975,Watermarked +3943,0.55,0.4151639344262295,-5.271696829221992,0.013305730606527047,Watermarked +3944,0.6166666666666667,0.4810792349726776,-3.1931249674651547,0.04959217968663012,Watermarked +3945,0.5901639344262295,0.42500000000000004,-4.904097017214922,0.01623024607177803,Watermarked +3946,0.5,0.3939890710382513,-2.375708105544391,0.09799696855032386,Watermarked +3947,0.6,0.448292349726776,-5.984088746615719,0.00934226482517408,Watermarked +3948,0.5081967213114754,0.42083333333333334,-2.548903055279766,0.08402331367690975,Watermarked +3949,0.6065573770491803,0.37499999999999994,-6.875471004283952,0.006301448294158944,Watermarked +3950,0.6229508196721312,0.375,-8.04851009961339,0.004005903148525875,Watermarked +3951,0.6166666666666667,0.43565573770491806,-8.810845426280132,0.00308060726714541,Watermarked +3952,0.5901639344262295,0.44583333333333336,-3.786983637511313,0.03228933418227714,Watermarked +3953,0.5333333333333333,0.4066256830601093,-3.9383341934243377,0.0291685537410846,Watermarked +3954,0.65,0.4560792349726776,-5.067243916956274,0.014838352365228525,Watermarked +3955,0.6,0.41058743169398904,-4.106153946552308,0.026146933323788254,Watermarked +3956,0.6065573770491803,0.3666666666666667,-11.1491127146561,0.0015463681392427176,Watermarked +3957,0.5901639344262295,0.4,-10.563455564617959,0.0018122415508561155,Watermarked +3958,0.5409836065573771,0.3625,-5.013582256122774,0.015278375182884808,Watermarked +3959,0.6557377049180327,0.38333333333333336,-4.064948814456438,0.026850007364746683,Watermarked +3960,0.55,0.3569672131147541,-7.573495790587439,0.004775004170929797,Watermarked +3961,0.5573770491803278,0.32500000000000007,-3.590011324398241,0.03702281920404291,Watermarked +3962,0.7213114754098361,0.4666666666666667,-4.791779056643048,0.017288564339617386,Watermarked +3963,0.5666666666666667,0.4355191256830601,-2.7776540598466357,0.06912570353142605,Watermarked +3964,0.5245901639344263,0.325,-7.221443843083816,0.005475227116046089,Watermarked +3965,0.5833333333333334,0.4687841530054645,-6.1047509028321505,0.008831382943901647,Watermarked +3966,0.6229508196721312,0.4041666666666667,-12.735108311334004,0.0010444936119927254,Watermarked +3967,0.6065573770491803,0.4458333333333333,-4.993743185709583,0.015445355698632398,Watermarked +3968,0.6333333333333333,0.43920765027322406,-3.6522403422007157,0.0354367438599517,Watermarked +3969,0.6166666666666667,0.3688524590163934,-6.85232843082187,0.006362416633878472,Watermarked +3970,0.5,0.38558743169398907,-3.0561029466886156,0.05516204355542588,Watermarked +3971,0.5666666666666667,0.360724043715847,-8.874674345400543,0.0030165490767111924,Watermarked +3972,0.5333333333333333,0.38642987249544625,-3.3493937263441476,0.07875291250488248,Watermarked +3973,0.6333333333333333,0.4558743169398907,-3.7027285641405947,0.03421325208693942,Watermarked +3974,0.45901639344262296,0.35833333333333334,-2.0822726427097917,0.12872281264009697,Watermarked +3975,0.7166666666666667,0.42691256830601093,-7.141009375063902,0.005653944218972081,Watermarked +3976,0.6,0.42704918032786887,-3.8405509086761342,0.031137895819023056,Watermarked +3977,0.5333333333333333,0.44808743169398907,-2.524971074527996,0.08580160862644565,Watermarked +3978,0.6721311475409836,0.5,-4.543656514596028,0.019965007637516786,Watermarked +3979,0.6,0.5059426229508197,-2.819285590304159,0.06677779498737416,Watermarked +3980,0.6229508196721312,0.4125,-5.2754090950771,0.01327985815124129,Watermarked +3981,0.5,0.3941256830601093,-9.09710470999564,0.002806625908000589,Watermarked +3982,0.7049180327868853,0.35833333333333334,-9.378324457589393,0.002568020999171169,Watermarked +3983,0.7049180327868853,0.3125,-10.826994016721436,0.0016856521372314135,Watermarked +3984,0.6833333333333333,0.49767759562841535,-7.0718291988043145,0.005813854687167852,Watermarked +3985,0.5409836065573771,0.38333333333333336,-3.5333558289091114,0.0385464838592851,Watermarked +3986,0.5833333333333334,0.4810792349726776,-2.348566398200449,0.10043648507703083,Watermarked +3987,0.5666666666666667,0.4560109289617486,-2.3723659709016056,0.09829344194941916,Watermarked +3988,0.55,0.41454918032786886,-2.568978311749152,0.08256644079807948,Watermarked +3989,0.7049180327868853,0.4083333333333333,-6.76630498961627,0.006595971681668088,Watermarked +3990,0.43333333333333335,0.35252732240437157,-2.7722967072440508,0.06943525700132411,Watermarked +3991,0.6333333333333333,0.43968579234972677,-6.462196948447438,0.0075180098331947,Watermarked +3992,0.6229508196721312,0.3958333333333333,-4.368818491249194,0.02218094363638083,Watermarked +3993,0.6666666666666666,0.4734289617486339,-4.251277919668526,0.023852073711438902,Watermarked +3994,0.5737704918032787,0.43333333333333335,-20.63996275066481,0.00024870591964064987,Watermarked +3995,0.5333333333333333,0.40232240437158473,-2.8318259683509934,0.06609008772343258,Watermarked +3996,0.5666666666666667,0.410655737704918,-3.6504320440032956,0.03548159461405821,Watermarked +3997,0.55,0.44795081967213113,-2.5131484576925534,0.08669715647138371,Watermarked +3998,0.6065573770491803,0.38333333333333336,-4.2005187653190665,0.02462402343066429,Watermarked +3999,0.639344262295082,0.4041666666666667,-8.814856757089686,0.003076528811000136,Watermarked diff --git a/papers/atharva_rasane/banner.png b/papers/atharva_rasane/banner.png new file mode 100644 index 0000000000..e6a793bd6c Binary files /dev/null and b/papers/atharva_rasane/banner.png differ diff --git a/papers/atharva_rasane/main.md b/papers/atharva_rasane/main.md new file mode 100644 index 0000000000..19ec32e256 --- /dev/null +++ b/papers/atharva_rasane/main.md @@ -0,0 +1,786 @@ +--- +# Ensure that this title is the same as the one in `myst.yml` +title: AI-Driven Watermarking Technique for Safeguarding Text Integrity in the Digital Age +abstract: | + The internet's growth has led to a surge in text usage. Now, with public access to generative AI models like ChatGPT/Bard, identifying the source is vital. This is crucial due to concerns about copyright infringement and plagiarism. Moreover, it is essential to differentiate AI-generated text to curb misinformation from AI model hallucinations. + + In this paper, we explore text watermarking as a potential solution, focusing on plain ASCII text in English. We investigate techniques including physical watermarking (e.g., UniSpaCh by Por et al.), which modifies text to hide a binary message using Unicode Spaces, and logical watermarking (e.g., word context by Jalil et al.), which generates a watermark key via a defined process. While logical watermarking is difficult to break but undetectable without prior knowledge, physical watermarks are easily detected but also easy to break. + + This paper presents a unique physical watermarking technique based on word substitution to address these challenges. The core idea is that AI models consistently produce the same output for the same input. Initially, we replaced every i-th word (for example, every 5th word) with a "[MASK]," a placeholder token used in natural language processing models to indicate where a word has been removed and needs to be predicted. Then, we used a BERT model to predict the most probable token in place of "[MASK]." The resulting text constitutes the watermarked text. To verify, we reran the algorithm on the watermarked text and compared the input and output for similarity. + + The Python implementation of the algorithm in this paper employs models from the HuggingFace Transformer Library, namely "bert-base-uncased" and "distilroberta-base". The "[MASK]" placeholder was generated by splitting the input string using the split() function and then replacing every 5th element in the list with "[MASK]". This modified list served as the input text for the BERT model, where the output corresponding to each "[MASK]" was replaced accordingly. Finally, applying the join() function to the list produces the watermarked text. + + This technique tends to generate nearly invisible watermarked text, preserving its integrity or completely changing the meaning of the text based on how similar the text is to the training dataset of BERT. This was observed when the algorithm was run on the story of Red Riding Hood, where its meaning was altered. However, the nature of this watermark makes it extremely difficult to break due to the black-box nature of the AI model. +--- + +## Introduction +The growth of the internet is driven by the spread of web pages, which are written in HTML (Hyper Text Markup Language). These web pages contain large amounts of text. Almost every webpage, in some form or another, contains text, making it a popular mode of communication, whether it be blogs, posts, articles, comments, etc. Text can be represented as a collection of ASCII or Unicode values, where each value corresponds to a specific character. Given the text-focused nature of the internet and tools like ChatGPT or Bard, it is crucial to identify the source of text. This helps to manage copyright issues and distinguish between AI-generated and human-written text, thereby preventing the spread of misinformation. Currently, detecting AI-generated text relies on machine learning classifiers that need frequent retraining with the latest AI-generated data. However, this method has drawbacks, such as the rapid evolution of AI models producing increasingly human-like text. Therefore, a more stable approach is needed, one that does not depend on the specific AI model generating the text. + +Watermarks are an identifying pattern used to trace the origin of the data. In this case, we specifically want to focus on text watermarking (watermarking of plain text). Text watermarking can broadly be classified into 2 types, Logical Embedding, and Physical Embedding, which in turn can be classified further [@Atr01]. Logical Embedding involves the user generating a watermark key by some logic from the input text. Note that this means that the input text is not altered, and the user instead keeps the generated watermark key to identify the text. Physical Embedding involves the user altering the input text itself to insert a message into it, and the user instead runs an algorithm to find this message to identify the text. In this paper, we will propose an algorithm to watermark text using BERT (Bidirectional Encoder Representations from Transformers), a model introduced by Google, whose main purpose is to replace a special symbol "[MASK]" with the most probable word given the context. + +BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained model introduced by Google in 2018, which has revolutionized natural language processing (NLP) [@Atr03]. "Pre-trained" means the model has already been trained on a large dataset before being fine-tuned for specific tasks. This allows the model to learn general features and patterns from a broad range of text data. For BERT, this pre-training involves vast amounts of text from books, articles, and websites, enabling it to understand the intricacies of human language. This pre-training allows BERT to be adapted quickly to various NLP tasks with relatively small amounts of task-specific data. Traditional models read text sequentially, either left-to-right or right-to-left. In contrast, BERT reads text in both directions simultaneously, providing a deeper understanding of context and meaning. This bidirectional approach allows BERT to perform exceptionally well in various NLP tasks, including question answering, text classification, and named entity recognition. By grasping the nuances of language more effectively, BERT sets a new standard for accuracy and efficiency in NLP applications [@Atr03]. + +At its core, BERT employs a bi-directional Transformer encoder, which helps understand the relationships between words in a sentence. This enhances its comprehension of text by understanding context from both directions simultaneously. BERT undergoes pre-training through two tasks: Masked Language Modeling (MLM), where certain words in a sentence are masked and the model predicts them based on surrounding words, and Next Sentence Prediction (NSP), which involves determining if one sentence logically follows another. This comprehensive training enables BERT to excel in numerous NLP applications like question answering, text classification, and named entity recognition. Given its deep understanding of context and semantics, BERT is highly relevant to text watermarking. Watermarking text involves embedding identifying patterns within the text to trace its origin, which can be critical for copyright protection and distinguishing between AI-generated and human-written content. BERT's sophisticated handling of language makes it ideal for embedding watermarks in a way that is subtle yet robust, ensuring that the text remains natural while the watermark is detectable. This capability provides a more stable and reliable method for watermarking text, irrespective of the model generating the text, therefore offering a concrete solution amidst the evolving landscape of AI-generated content. + +## Related Work +In this section, we will review two text watermarking algorithms before introducing our proposed technique. Let's first look at the current standards for text watermarking. Text watermarking algorithms embed unique identifiers in text to protect copyright and verify authenticity. They are important because they help prevent unauthorized use, copying, and distribution of text. + +The first algorithm is Word Context, developed by Jalil & Mirza in 2009. It is a type of logical watermarking that generates a watermark key without altering the original text [@Proc01]. Logical watermarking involves embedding a watermark key without changing the original text. Word Context generates a watermark key by analyzing the structure of the text around selected keywords and creating a pattern based on word lengths [@Proc01]. In Word Context, a keyword is selected. For example, using the keyword 'is' in the text 'Pakistan is a developing country, with Islamabad is the capital of Pakistan. It is located in Asia.' The lengths of the words before and after 'is' are recorded: 'Pakistan' (8) and 'a' (1), 'Islamabad' (9) and 'the' (3), 'It' (2) and 'located' (7). The watermark is then 8-1-9-3-2-7 [@Proc01]. The keyword is chosen based on its significance in the text. Word lengths are used to create the watermark because they provide a unique pattern without altering the text, ensuring the watermark is imperceptible [@Proc01]. + +The second algorithm, UniSpaCh by Kamaruddin et al. in 2018, modifies the white spaces in text to embed a binary message directly into it [@Atr04]. Modifying white spaces changes the spacing patterns in the text, embedding binary information. A binary message is a sequence of bits (0s and 1s) that represents data. This method uses different types of spaces to encode these bits [@Atr04]. UniSpaCh uses 2-bit categorization to create a binary string (e.g., '10', '01', '00', '11'). Each pair of bits is replaced with a unique type of space (like punctuation space, thin space). These spaces are then placed in areas like between words, sentences, and paragraphs. This method is highly invisible but has low capacity, making it unsuitable for embedding long messages [@Atr04]. 2-bit categorization assigns pairs of bits to specific types of spaces. This method is considered invisible because the changes are subtle and not easily noticeable by readers. It has low capacity because only a few bits can be embedded per space, limiting the amount of information that can be hidden [@Atr04]. + +The first approach by Jalil & Mirza (2009) is not suitable for today's fast-paced generation of AI text, as it is impractical to store a logical watermark for each new text [@Proc01]. It is impractical to store a logical watermark for each text because the volume of generated text is too high, making it difficult to manage and store all watermarks. AI text generation has made it easier and faster to produce large amounts of text, increasing the need for scalable watermarking solutions [@Proc01]. The second approach by Por et al. (2012) is also not suitable because the watermark can be easily removed by reformatting the text. We need a robust and imperceptible watermarking technique [@Atr04]. The watermark can be removed by reformatting because changes in text layout, such as altering spaces or reformatting paragraphs, can disrupt the embedded watermark. A robust watermarking technique can withstand such changes and remain detectable, while an imperceptible technique ensures the watermark is not noticeable to the reader [@Atr04]. + +Our proposed technique is based on a method by Lancaster (2023) for ChatGPT [@Atr02]. It replaces every fifth word in a sequence of five consecutive words (non-overlapping 5-gram) with a word generated using a fixed random seed. For example, in the sentence 'The friendly robot greeted the visitors with a cheerful beep and a wave of its metal arms,' the non-overlapping 5-grams are 'The friendly robot greeted the,' 'visitors with a cheerful beep,' and 'and a wave of its metal.' We replace the words 'the,' 'visitors,' and 'metal' with words generated by ChatGPT using a fixed random seed [@Atr02]. A non-overlapping 5-gram is a sequence of five consecutive words without any overlap. Replacing every fifth word embeds the watermark without altering the overall meaning of the text, making it a subtle and effective method for embedding the watermark [@Atr02]. + +We check the watermark using overlapping 5-grams, which overlap by four words. For example, 'The friendly robot greeted the,' 'friendly robot greeted the visitors,' 'robot greeted the visitors with,' etc. This method uses ChatGPT to watermark its own text, but it requires running two ChatGPT models to ensure consistency across different outputs from the same seed. Overlapping 5-grams are sequences of five words that overlap by four words. Two models of ChatGPT are needed to ensure consistent watermarking across different outputs because different models might produce different results with the same random seed, and consistency is crucial for verifying the watermark. + +We propose using BERT, a model designed to find missing words, as a better alternative to ChatGPT. BERT is more precise and smaller. Its bidirectional nature uses more context for word prediction, potentially leading to better results. While ChatGPT-based algorithms are best for ChatGPT text, BERT can be used for any text, regardless of its origin. BERT is better than ChatGPT for this purpose because it is more precise and smaller, making it more efficient. BERT's bidirectional nature means it uses context from both the preceding and following words to predict a missing word, which can lead to more accurate results. + + +## Proposed Model +"BERT-based watermarking is based on the 5-gram approach by Lancaster[@Atr02]. However, our focus is on watermarking any text, regardless of its origin. This paper will use **bert-base-uncased** model, which finds the most probable uncased English word to replace the [MASK] token. + +Note that a different variant of BERT can be trained on different language datasets and thus will generate a different result and as such the unique identity to consider here is the BERT model i.e. if the user wants a unique watermark they need to train/develop the BERT model on their own. This paper is not concerned with the type of BERT model and is focused on its conceptual application for watermarking. Thus for us, BERT is a black box model that returns the most probable word given the context with the only condition being that it has a constant temperature i.e. it does not hallucinate (produce different results for the same input). For our purposes, you can think of the proposed algorithm as a many to one function which is responsible for converting the input text into a subset of watermarked set. + +## Algorithm +**Watermark Encoding** +:::{figure} Algorithm-Encoding.png +:label: fig:1 +Encoding algorithm to watermark input text +::: + +The above is a simple implementation of the algorithm where we are assuming +1. The only white spaces in the text are " ". +2. BERT model has infinite context. + +This simplified code allows us to grasp the core of the algorithm. First, we split the input text into a list of words using the split() function. Next, we replace every 5th word with the string "[MASK]" which represents a special token indicating where BERT should predict a word. For each [MASK] token, we pass the preceding words and the 4 following words to the BERT model, assuming BERT can handle an infinite context. In reality, BERT has a limited context, so we use up to maximum_context_size - 5 words along with the [MASK] token. The missing_word_form_BERT() function returns the most probable word, which replaces the [MASK] token in the list. We continue this process until all [MASK] tokens are replaced, then convert the list of words back into a string using " ".join() + +The beauty of the algorithm is that if we were to run it again on the watermarked text the output that we would get would be the same as the input thus to check if a given text is watermarked we simply need to compare the input and output to determine if a given text is watermarked we simply need to run the above algorithm again, but with a few changes we will have to take in offset as a consideration as the one plagiarizing the text might insert additional words that may lead to the text + +**Watermark Detection** + +The algorithm checks if a given text is watermarked by comparing the input and output texts, considering possible word insertions that may offset the watermark pattern. + +1. **Input Text Preparation :** Obtain the suspected watermarked text as input. + +2. **Run Watermark Detection Algorithm:** Run the watermark detection algorithm on the input text. + +3. **Compare Input and Output:** If the input matches the output, the text is watermarked.If not, proceed to check with offsets. + +4. **Offset Consideration:** Initialize an array to store match percentages for each offset: `offsets = [0, 1, 2, 3, 4]`.For each offset, adjust the input text by removing `n % 5` words where `n` is the number of words added. + +5. **Check for Matches:** For each offset, count the matches where the watermark pattern (every 5th word replaced) aligns. + +6. **Store Match Percentages:** Calculate the percentage of matches for each offset and store them. + +7. **Statistical Analysis:** Compute the highest percentage of matches (`Highest Ratio`). Compute the average percentage of matches for the remaining offsets (`Average Others`). Calculate the T-Statistic and P-Value to determine the statistical difference between `Highest Ratio` and `Average Others`. The T-Statistic measures the difference between groups, and the P-Value indicates the significance of this difference. + +8. **Classification:** Use a pre-trained model to classify the text based on the metrics (`Highest Ratio`, `Average Others`, T-Statistic, P-Value) as watermarked or not. + + +## Implementation - Encoding module +Let's examine a Python implementation of the proposed watermarking model. The watermark_text module identifies every 5th word in the input string, splits them using Python's built-in split() function, and marks them for modification using BERT. These placeholders are replaced with the [MASK] token. Although we use BERT here, the module can be adapted to other AI models. We chose BERT due to its efficiency in altering individual words. The 5th word is selected to ensure a consistent and detectable pattern. +**The choice of index = 5 is because?** +**Also is this code picked from a paper?** + +```python +import os +os.environ['HUGGINGFACEHUB_API_TOKEN'] = '' +from transformers import pipeline, AutoTokenizer, AutoModelForMaskedLM +import torch + +def watermark_text(text, model_name="bert-base-uncased", offset=0): + # Clean and split the input text + text = " ".join(text.split()) + words = text.split() + + # Replace every fifth word with [MASK], starting from the offset + for i in range(offset, len(words)): + if (i + 1 - offset) % 5 == 0: + words[i] = '[MASK]' + + # Initialize the tokenizer and model, move to GPU if available + device = 0 if torch.cuda.is_available() else -1 + tokenizer = AutoTokenizer.from_pretrained(model_name) + model = AutoModelForMaskedLM.from_pretrained(model_name).to(device) + + # Initialize the fill-mask pipeline + classifier = pipeline("fill-mask", model=model, tokenizer=tokenizer, device=device) + + # Make a copy of the words list to modify it + watermarked_words = words.copy() + + # Process the text in chunks + for i in range(offset, len(words), 5): + chunk = " ".join(watermarked_words[:i+9]) + if '[MASK]' in chunk: + try: + tempd = classifier(chunk) + except Exception as e: + print(f"Error processing chunk '{chunk}': {e}") + continue + + if tempd: + templ = tempd[0] + temps = templ['token_str'] + watermarked_words[i+4] = temps.split()[0] + + return " ".join(watermarked_words) + +# Example usage +text = "Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are infeasible for classical computers. Unlike classical computers, which use bits as the fundamental unit of information, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously due to the principles of superposition and entanglement, providing a significant advantage in solving complex computational problems." +watermark_text(text, offset=0) +result = "Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are impossible for classical computers. Unlike quantum computers, which use bits as the fundamental unit of , quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously according to the principles of symmetry and entanglement, providing a significant advantage in solving complex mathematical problems." +``` +In the result, the module has replaced each 5th word with the most probable replacement word selected by BERT. There will always be some words that AI would not alter. For example the 10th word "the" and the 15th word "to". These cannot be changed by AI without altering the entire sentence. +Further, to speed up the AI computing, we can employ GPUs in this module as well as the Detection module. + +## Implementation - Detection Module +Now that we have our watermarked text, we need to identify potential copyright infringement. We assume this text is what a plagiarizer has access to. + +For this, we create a module to check the number of word matches if the AI model with the same offset parameter is run on the watermarked text again. The algorithm's elegance lies in its consistency: if we run it again on the watermarked text, the output will match the input because the most probable words are already present at every 5th offset. Consequently, we get a 100% match rate with a match ratio of 1. If all the 5th words were altered, our match rate would be 0. + +Altering written text is a posibility we cannot ignore. consider a scenario where a plagiarizer might insert extra words, causing the input not to match the output exactly. This means our model needs to check for watermarks not only at a specific index but also in the surrounding words. Therefore, our model needs to check for the watermark at different offsets (0 to 4) to account for potential word insertions. + +Here is how the offset works: +- If 1 word is added at the start, the offset is 1. +- If 2 words are added, the offset is 2. +- If 3 words are added, the offset is 3. +- If 4 words are added, the offset is 4. +- If 5 words are added, the offset is 0 (since the algorithm replaces every 5th word). + +In general, if 'n' words are added, the offset is `n % 5`. Since we do not know how many words were added, we need to check all possible offsets (0, 1, 2, 3, 4). + +If words are added in the middle of the text, the majority of the watermark pattern (every 5th word replaced) will still be detectable at some offset. The idea is that one offset will show a higher number of matches compared to others, indicating a watermark. + +For detection, we store the percentage of matches for each offset. There is no fixed threshold for determining a watermark, as the choice of words affects the number of matches. For non-watermarked text, the percentage of matches at each offset will be similar. For watermarked text, one offset will have a significantly higher percentage of matches. + +The output of "watermark_text_and_calculate_matches" module is a match ratio for offsets 0-4 acting as a seed for the next stage of detection. Below is the python code for generating the list of match ratios. + +```python +def watermark_text_and_calculate_matches(text, model_name="bert-base-uncased", max_offset=5): + # Clean and split the input text + text = " ".join(text.split()) + words = text.split() + + # Initialize the tokenizer and model, move to GPU if available + device = 0 if torch.cuda.is_available() else -1 + tokenizer = AutoTokenizer.from_pretrained(model_name) + model = AutoModelForMaskedLM.from_pretrained(model_name).to(device) + + # Initialize the fill-mask pipeline + classifier = pipeline("fill-mask", model=model, tokenizer=tokenizer, device=device) + + # Dictionary to store match ratios for each offset + match_ratios = {} + + # Loop over each offset + for offset in range(max_offset): + # Replace every fifth word with [MASK], starting from the offset + modified_words = words.copy() + for i in range(offset, len(modified_words)): + if (i + 1 - offset) % 5 == 0: + modified_words[i] = '[MASK]' + + # Make a copy of the modified words list to work on + watermarked_words = modified_words.copy() + total_replacements = 0 + total_matches = 0 + + # Process the text in chunks + for i in range(offset, len(modified_words), 5): + chunk = " ".join(watermarked_words[:i+9]) + if '[MASK]' in chunk: + try: + tempd = classifier(chunk) + except Exception as e: + print(f"Error processing chunk '{chunk}': {e}") + continue + + if tempd: + templ = tempd[0] + temps = templ['token_str'] + original_word = words[i+4] + replaced_word = temps.split()[0] + watermarked_words[i+4] = replaced_word + + # Increment total replacements and matches + total_replacements += 1 + if replaced_word == original_word: + total_matches += 1 + + # Calculate the match ratio for the current offset + if total_replacements > 0: + match_ratio = total_matches / total_replacements + else: + match_ratio = 0 + + match_ratios[offset] = match_ratio + + # Return the match ratios for each offset + return match_ratios + +# Example usage +text = "Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are infeasible for classical computers. Unlike classical computers, which use bits as the fundamental unit of information, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously due to the principles of superposition and entanglement, providing a significant advantage in solving complex computational problems." + +# Calculate match ratios +match_ratios = watermark_text_and_calculate_matches(text, max_offset=5) +# (result rounded) match_ratio = {0: 0.54, 1: 0.62, 2: 0.58, 3: 0.67, 4: 0.58} +``` + +The final stage of detection involves determining if the match ratios are statistically significant. +To determine whether the text is watermarked, we rely on a binary classification of whether a text is watermarked. For this, we use a pre-trained model based on metrics including Highest Ratio, Average Others, T-Statistic, and P-Value. This approach is necessary because, as illustrated in the graphs later, there is no discernible or easily observable difference between the T-statistics and P-values of watermarked and non-watermarked texts. Consequently, we resort to using a pre-trained model for classification, which has achieved the highest accuracy of 94%. + +The module "check_significant_difference" generates a list of significance. + +```python +from scipy.stats import ttest_1samp +import numpy as np + +def check_significant_difference(match_ratios): + # Extract ratios into a list + ratios = list(match_ratios.values()) + + # Find the highest ratio + highest_ratio = max(ratios) + + # Find the average of the other ratios + other_ratios = [r for r in ratios if r != highest_ratio] + average_other_ratios = np.mean(other_ratios) + + # Perform a t-test to compare the highest ratio to the average of the others + t_stat, p_value = ttest_1samp(other_ratios, highest_ratio) + + # Print the results + print(f"Highest Match Ratio: {highest_ratio}") + print(f"Average of Other Ratios: {average_other_ratios}") + print(f"T-Statistic: {t_stat}") + print(f"P-Value: {p_value}") + + # Determine if the difference is statistically significant (e.g., at the 0.05 significance level) + if p_value < 0.05: + print("The highest ratio is significantly different from the others.") + else: + print("The highest ratio is not significantly different from the others.") + + return [highest_ratio, average_other_ratios, t_stat, p_value] + +# Example usage +text = "Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are infeasible for classical computers. Unlike classical computers, which use bits as the fundamental unit of information, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously due to the principles of superposition and entanglement, providing a significant advantage in solving complex computational problems." +match_ratios = watermark_text_and_calculate_matches(text, max_offset=5) +check_significant_difference(match_ratios) + +``` +The module "randomly_add_words" was created to simulate the scenario where additional words have been added to the watermarked test for testing purposes. + +```python +import random + +def randomly_add_words(text, words_to_add, num_words_to_add): + # Clean and split the input text + text = " ".join(text.split()) + words = text.split() + + # Insert words randomly into the text + for _ in range(num_words_to_add): + # Choose a random position to insert the word + position = random.randint(0, len(words)) + # Choose a random word to insert + word_to_insert = random.choice(words_to_add) + # Insert the word at the random position + words.insert(position, word_to_insert) + + # Join the list back into a string and return the modified text + modified_text = " ".join(words) + return modified_text + +# Example usage +text = "Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are infeasible for classical computers. Unlike classical computers, which use bits as the fundamental unit of information, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously due to the principles of superposition and entanglement, providing a significant advantage in solving complex computational problems." +words_to_add = ["example", "test", "random", "insert"] +num_words_to_add = 5 + +# modified_text = randomly_add_words(text, words_to_add, num_words_to_add) +modified_text = randomly_add_words(watermark_text(text, offset=0), words_to_add, num_words_to_add) +(result) modified_text = "Quantum computing is example a rapidly evolving field that leverages the principles of quantum mechanics to perform random computations that are impossible for classical computers. Unlike quantum computers, which use bits as the random insert fundamental unit of , quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously according random to the principles of symmetry and entanglement, providing a significant advantage in solving complex mathematical problems." + +match_ratios = watermark_text_and_calculate_matches(modified_text, max_offset=5) +# (result rounded) match_ratios = {0: 0.57, 1: 0.57, 2: 0.54, 3: 0.38, 4: 0.77} + +check_significant_difference(match_ratios) +# (result rounded) +# Highest Match Ratio: 0.77 +# Average of Other Ratios: 0.52 +# T-Statistic: -5.66 +# P-Value: 0.01 +# The highest ratio is significantly different from the others. +``` + +Once the list of significance is defined, to show the significance of using a pre-trained model, lets plot them to futher understand the statistical summary. Here is the python code used to generate the plots. + +```python +import pandas as pd +import seaborn as sns +import matplotlib.pyplot as plt +from scipy.stats import ttest_ind +from sklearn.model_selection import train_test_split +from sklearn.ensemble import RandomForestClassifier +from sklearn.metrics import classification_report, confusion_matrix + +# Assuming list_of_significance and list_of_significance_watermarked are already defined +# Create DataFrames from the lists +df_significance = pd.DataFrame(list_of_significance, columns=['Highest Ratio', 'Average Others', 'T-Statistic', 'P-Value']) +df_significance_watermarked = pd.DataFrame(list_of_significance_watermarked, columns=['Highest Ratio', 'Average Others', 'T-Statistic', 'P-Value']) + +# Add a label column to distinguish between the two sets +df_significance['Label'] = 'Original' +df_significance_watermarked['Label'] = 'Watermarked' + +# Combine the DataFrames +combined_df = pd.concat([df_significance, df_significance_watermarked], ignore_index=True) + +# Perform EDA +def perform_eda(df): + # Display the first few rows of the DataFrame + print("First few rows of the DataFrame:") + print(df.head()) + + # Display statistical summary + print("\nStatistical Summary:") + print(df.describe()) + + # Check for missing values + print("\nMissing Values:") + print(df.isnull().sum()) + + # Visualize the distributions of the features + plt.figure(figsize=(12, 8)) + sns.histplot(data=df, x='Highest Ratio', hue='Label', element='step', kde=True) + plt.title('Distribution of Highest Ratio') + plt.show() + + plt.figure(figsize=(12, 8)) + sns.histplot(data=df, x='Average Others', hue='Label', element='step', kde=True) + plt.title('Distribution of Average Others') + plt.show() + + plt.figure(figsize=(12, 8)) + sns.histplot(data=df, x='T-Statistic', hue='Label', element='step', kde=True) + plt.title('Distribution of T-Statistic') + plt.show() + + plt.figure(figsize=(12, 8)) + sns.histplot(data=df, x='P-Value', hue='Label', element='step', kde=True) + plt.title('Distribution of P-Value') + plt.show() + + # Pairplot to see relationships + sns.pairplot(df, hue='Label') + plt.show() + + # Correlation matrix + plt.figure(figsize=(10, 8)) + sns.heatmap(df.drop(columns=['Label']).corr(), annot=True, cmap='coolwarm') + plt.title('Correlation Matrix') + plt.show() + + # T-test to check for significant differences + original = df[df['Label'] == 'Original'] + watermarked = df[df['Label'] == 'Watermarked'] + + for column in ['Highest Ratio', 'Average Others', 'T-Statistic', 'P-Value']: + t_stat, p_value = ttest_ind(original[column], watermarked[column]) + print(f"T-test for {column}: T-Statistic = {t_stat}, P-Value = {p_value}") + +# Perform EDA on the combined DataFrame +perform_eda(combined_df) + +# Check if the data is ready for machine learning classification + +# Prepare the data +X = combined_df.drop(columns=['Label']) +y = combined_df['Label'] + +# Convert labels to numerical values for ML model +y = y.map({'Original': 0, 'Watermarked': 1}) + +# Split the data into training and testing sets +X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) + +# Train a RandomForestClassifier +clf = RandomForestClassifier(random_state=42) +clf.fit(X_train, y_train) + +# Make predictions +y_pred = clf.predict(X_test) + +# Evaluate the model +print("\nClassification Report:") +print(classification_report(y_test, y_pred)) + +print("\nConfusion Matrix:") +print(confusion_matrix(y_test, y_pred)) + +# Feature importances +feature_importances = clf.feature_importances_ + +# Create a DataFrame for feature importances +feature_importances_df = pd.DataFrame({ + 'Feature': X.columns, + 'Importance': feature_importances +}).sort_values(by='Importance', ascending=False) + +# Plot feature importances +plt.figure(figsize=(12, 8)) +sns.barplot(x='Importance', y='Feature', data=feature_importances_df, palette='viridis') +plt.title('Feature Importances') +plt.show() + +# Heatmap for feature importances +plt.figure(figsize=(10, 8)) +sns.heatmap(feature_importances_df.set_index('Feature').T, annot=True, cmap='viridis') +plt.title('Heatmap of Feature Importances') +plt.show() +``` + +The plots are created using the result from our previous example with check_significant_difference returned: + Highest Match Ratio: 0.7692307692307693 + Average of Other Ratios: 0.5164835164835164 + T-Statistic: -5.66220858504931 + P-Value: 0.010908789440745323 + +```{list-table} First few rows of the DataFrame +:label: tbl:Dataframe +:header-rows: 1 +* - + - Highest Ratio + - Average Others + - T-Statistic + - P-Value + - Label +* - 0 + - 0.233333 + - 0.182203 + - -3.532758 + - 0.038563 + - Original +* - 1 + - 0.203390 + - 0.139195 + - -3.440591 + - 0.041218 + - Original +* - 2 + - 0.338983 + - 0.270339 + - -2.228608 + - 0.112142 + - Original +* - 3 + - 0.254237 + - 0.168362 + - -2.451613 + - 0.246559 + - Original +* - 4 + - 0.288136 + - 0.210876 + - -5.467540 + - 0.012026 + - Original +``` +```{list-table} Statistical Summary +:label: tbl:Statistical_Summary +:header-rows: 1 +* - + - Highest Ratio + - Average Others + - T-Statistic + - P-Value +* - count + - 4000.000000 + - 4000.000000 + - 3999.000000 + - 3999.000000 +* - mean + - 0.490285 + - 0.339968 + - -6.076672 + - 0.036783 +* - std + - 0.128376 + - 0.082900 + - 5.580957 + - 0.043217 +* - min + - 0.101695 + - 0.066667 + - -111.524590 + - 0.000002 +* - 25% + - 0.416667 + - 0.296610 + - -6.938964 + - 0.006418 +* - 50% + - 0.491525 + - 0.354732 + - -4.431515 + - 0.021973 +* - 75% + - 0.573770 + - 0.398224 + - -3.176861 + - 0.052069 +* - max + - 0.868852 + - 0.580601 + - -1.166065 + - 0.451288 +``` + +Missing Values: +Highest Ratio 0 +Average Others 0 +T-Statistic 1 +P-Value 1 +Label 0 +dtype: int64 + +:::{figure} Distribution_of_highest_ratio.png +:label: fig:2 +Distribution of highest ratio +::: + +:::{figure} Distribution_of_average_others.png +:label: fig:3 +Distribution of average others +::: + +:::{figure} Distribution_of_t-statistics.png +:label: fig:4 +Distribution of t-statistics +::: + +:::{figure} Distribution_of_P-value.png +:label: fig:5 +Distribution of P-value +::: + +:::{figure} Dataset.png +:label: fig:6 +Dataset +::: + +:::{figure} Correlation_Matrix.png +:label: fig:7 +Correlation_Matrix +::: + +From the graphs and statistical summaries, several inferences can be drawn regarding the distributions and relationships between the variables in the dataset: + +**Distribution of Highest Ratio:** +The distribution of the "Highest Ratio" variable shows a clear distinction between the "Original" and "Watermarked" categories. +The "Original" category has a peak around 0.4, while the "Watermarked" category peaks around 0.5, indicating a shift in the distribution towards higher values for the watermarked data. + +**Distribution of Average Others:** +Similarly, the "Average Others" variable shows a distinction between the two categories. +The "Original" category peaks around 0.3, whereas the "Watermarked" category peaks slightly higher, around 0.4. +This suggests that the average values for other ratios are higher in the watermarked data compared to the original data. +Distribution of T-Statistic: + +**Distribution of T-statistic:** +The distribution of the T-statistic is highly skewed to the left for both categories, with a long tail extending to very negative values. +The "Original" category appears to have a more pronounced peak near 0, while the "Watermarked" category has a lower count at the peak and a wider spread. + +**Distribution of P-Value:** +The P-value distribution is heavily skewed towards 0 for both categories, with the "Watermarked" category showing a sharper peak at 0. +This suggests that most of the tests result in very low p-values, indicating strong statistical significance in the differences observed. + +**Pair Plot:** +The pair plot provides a visual comparison of the relationships between the variables for the two categories. +There are clear clusters and separations between the "Original" and "Watermarked" categories in the scatter plots, particularly for "Highest Ratio" vs. "Average Others" and "Highest Ratio" vs. "P-Value". +This reinforces the idea that the watermarked data exhibits different characteristics compared to the original data. + +**Correlation Matrix:** +The correlation matrix shows the pairwise correlation coefficients between the variables. +"Highest Ratio" and "Average Others" are positively correlated (0.66), indicating that higher values of the highest ratio tend to be associated with higher average values of other ratios. +"T-Statistic" has a negative correlation with "Highest Ratio" (-0.35) and "P-Value" (-0.31), suggesting that higher ratios tend to result in more negative T-statistics and lower p-values. + +**Overall Observations:** +The "Watermarked" data tends to have higher ratios and averages compared to the "Original" data. +The T-statistics and p-values indicate strong statistical differences between the original and watermarked categories. +The pair plot and correlation matrix provide further evidence of distinct patterns and relationships in the watermarked data compared to the original data. + +While these plots do show a difference between the watermarked and non-watermarked text, using a pre-trained model help us achieve higher efficiency and consistency in our comparisons. + +## Model Training, Testing and Efficiency +The algorithm in this paper was trained using a dataset generated from Gutenberg's top 10 books [@book01], [@book02], [@book03], [@book04], [@book05], [@book06], [@book07], [@book08], [@book09], [@book10]. Specifically, 2000 random 300-word paragraphs were taken from these books, ensuring an equal number of paragraphs from each book. Each paragraph was watermarked, and then statistical analysis was performed. The Highest Match Ratio, Average of Other Ratios, T-Statistic, and P-Value were calculated and stored in Results.csv. The models were trained using an 80/20 split of the dataset, with the following models being trained: Logistic Regression, Decision Tree, Random Forest, Support Vector Machine, Gradient Boosting, AdaBoost, Naive Bayes, and K-Nearest Neighbors. Gradient Boosting gave the highest accuracy, resulting in an overall accuracy of 94% in identifying watermarked text + +**Code used for model training** +```python +import pandas as pd +import seaborn as sns +import matplotlib.pyplot as plt +from sklearn.model_selection import train_test_split +from sklearn.linear_model import LogisticRegression +from sklearn.tree import DecisionTreeClassifier +from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier, AdaBoostClassifier +from sklearn.svm import SVC +from sklearn.naive_bayes import GaussianNB +from sklearn.neighbors import KNeighborsClassifier +from sklearn.metrics import classification_report, confusion_matrix + +# Assuming list_of_significance and list_of_significance_watermarked are already defined +# Create DataFrames from the lists +df_significance = pd.DataFrame(list_of_significance, columns=['Highest Ratio', 'Average Others', 'T-Statistic', 'P-Value']) +df_significance_watermarked = pd.DataFrame(list_of_significance_watermarked, columns=['Highest Ratio', 'Average Others', 'T-Statistic', 'P-Value']) + +# Add a label column to distinguish between the two sets +df_significance['Label'] = 'Original' +df_significance_watermarked['Label'] = 'Watermarked' + +# Combine the DataFrames +combined_df = pd.concat([df_significance, df_significance_watermarked], ignore_index=True) +combined_df = combined_df.dropna() + +# Prepare the data +X = combined_df.drop(columns=['Label']) +y = combined_df['Label'] + +# Convert labels to numerical values for ML model +y = y.map({'Original': 0, 'Watermarked': 1}) + +# Split the data into training and testing sets +X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) + +# Initialize models +models = { + 'Logistic Regression': LogisticRegression(random_state=42, max_iter=1000), + 'Decision Tree': DecisionTreeClassifier(random_state=42), + 'Random Forest': RandomForestClassifier(random_state=42), + 'Support Vector Machine': SVC(random_state=42), + 'Gradient Boosting': GradientBoostingClassifier(random_state=42), + 'AdaBoost': AdaBoostClassifier(random_state=42), + 'Naive Bayes': GaussianNB(), + 'K-Nearest Neighbors': KNeighborsClassifier() +} + +# Train and evaluate models +for model_name, model in models.items(): + model.fit(X_train, y_train) + y_pred = model.predict(X_test) + print(f"\n{model_name} Classification Report:") + print(classification_report(y_test, y_pred)) + print(f"\n{model_name} Confusion Matrix:") + print(confusion_matrix(y_test, y_pred)) + + # Feature importances (only for models that provide it) + if hasattr(model, 'feature_importances_'): + feature_importances = model.feature_importances_ + feature_importances_df = pd.DataFrame({ + 'Feature': X.columns, + 'Importance': feature_importances + }).sort_values(by='Importance', ascending=False) + + # Plot feature importances + plt.figure(figsize=(12, 8)) + sns.barplot(x='Importance', y='Feature', data=feature_importances_df, palette='viridis') + plt.title(f'{model_name} Feature Importances') + plt.show() +``` + +**Code for Model testing** +```python +import os +import random + +def extract_test_cases(folder_path, num_cases=2000, words_per_case=300): + test_cases = [] + book_files = [f for f in os.listdir(folder_path) if os.path.isfile(os.path.join(folder_path, f))] + + # Calculate the number of test cases to extract from each book + cases_per_book = num_cases // len(book_files) + extra_cases = num_cases % len(book_files) + + for book_file in book_files: + with open(os.path.join(folder_path, book_file), 'r', encoding='utf-8') as file: + text = file.read() + words = text.split() + num_words = len(words) + + # Ensure enough words are available to extract the cases + if num_words < words_per_case: + continue + + # Determine the number of cases to extract from this book + num_cases_from_book = cases_per_book + if extra_cases > 0: + num_cases_from_book += 1 + extra_cases -= 1 + + for _ in range(num_cases_from_book): + start_index = random.randint(0, num_words - words_per_case) + case = ' '.join(words[start_index:start_index + words_per_case]) + test_cases.append(case) + + if len(test_cases) == num_cases: + return test_cases + + return test_cases + +# Usage example +folder_path = 'books' +test_cases = extract_test_cases(folder_path) +``` +```python +list_of_significance = [] +list_of_significance_watermarked = [] +count_t = 0 +for text in test_cases: + count_t+=1 + print("___________________________________________________________________________________________________________________________") + print("Doing", count_t) + print("___________________________________________________________________________________________________________________________") + + words_to_add = ["example", "test", "random", "insert"] + num_words_to_add = 5 + + modified_text = randomly_add_words(watermark_text(text, offset=0), words_to_add, num_words_to_add) + + match_ratios = watermark_text_and_calculate_matches(modified_text, max_offset=5) + list_of_significance_watermarked.append(check_significant_difference(match_ratios)) + + match_ratios = watermark_text_and_calculate_matches(text, max_offset=5) + list_of_significance.append(check_significant_difference(match_ratios)) + + print("___________________________________________________________________________________________________________________________") + print("Done", count_t, ) + print("___________________________________________________________________________________________________________________________") +``` + +## Analysis of the Algorithm + +**Strengths:** +1. Robustness against attacks: The BERT-based watermarking algorithm uses the sophisticated context-understanding capability of BERT to embed watermarks. This makes the watermark integration deeply intertwined with the text's semantic structure, which is difficult to detect and remove without altering the underlying meaning, thus providing robustness against simple text manipulation attacks. +2. Comparison with existing methods: Compared to traditional watermarking methods like word context and UniSpaCh, the BERT-based approach offers a more adaptable and less detectable method. It does not rely on altering visible text elements or patterns easily erased, like white spaces or specific word sequences. Instead, it uses semantic embedding, making it superior in maintaining the natural flow and readability of the text. +3. Scalability and adaptability: The method is scalable to different languages and text forms by adjusting the BERT model used. It can be adapted to work with different BERT variants trained on specific datasets, enhancing flexibility in deployment. + +**Challenges:** +1. Dependency on model consistency: The watermark detection relies heavily on the consistency of the BERT model's output. Any updates or changes in the model could potentially alter the watermark, making it undetectable. If the watermark can embbed some sort of version history and control, this could be managed. +2. Data Integrity is highly dependent on the Model: the integrity of the watermarked text depends on how good the model is at replacing the given word, due to the nature of AI-generated text where all the previous tokens are used to generate new ones BERT watermarking can preserve integrity much more effectively. However if it were to watermark text which is completely different from its training dataset it might return an incoherent output, for example if the dataset of BERT consists of scientific papers it will struggle immensely when trying to watermark fairy tails. +3. Potential for false positives/negatives: Given the probabilistic nature of BERT's predictions, there is a risk of incorrect watermark detection, especially in texts with complex semantics or those that closely mimic the watermark patterns without actually being watermarked. +4. Potential loss of context: When words are replaced, the intended context of delivery could be altered. However, AI models are continually improving, and we hope that a well-trained model can significantly mitigate this risk. + +**Real-world applicability:** +1. Versatility in applications: This method can be applied across various fields such as copyright protection, and content authentication, and in legal and academic settings where proof of authorship is crucial. It is particularly beneficial for managing copyrights in digital media, academic papers, and any online content where text is dynamically generated or reused. +2. Integration with existing systems: The algorithm can be seamlessly integrated with current content management systems (CMS) and digital rights management (DRM) systems, enhancing their capabilities to include advanced text watermarking features. This integration helps organizations maintain control over their content distribution and monitor usage without invasive methods. +3. Application in AI-generated text: With the proliferation of AI-generated content from models like ChatGPT, GPT-4, and other AI writing assistants, distinguishing between human-generated and AI-generated text becomes crucial. The BERT-based watermarking can be used to embed unique, non-intrusive identifiers into AI-generated texts, ensuring that each piece of content can be traced back to its source. This is particularly valuable in preventing the spread of misinformation, verifying the authenticity of content, and in applications where copyright claims on AI-generated content might be disputed. +4. Forensic Linguistics in Cybersecurity: In cybersecurity, determining the origin of phishing emails or malicious texts can be crucial. BERT-based watermarking can assist forensic linguists and security professionals by providing a means to trace the origins of specific texts back to their creators, helping to identify patterns or sources of cyber threats. +5. Enhanced Licensing Control for Digital Text: As digital content licensing becomes more complex with different rights for different geographies and platforms, watermarking can help content owners and licensing agencies enforce these rights more effectively. The watermark makes it easier to enforce and monitor compliance automatically. + + +## Conclusion +By leveraging the BERT model and the proposed algorithm, we have achieved a 94% accuracy rate in detecting watermarked text. With an appropriate training dataset and ongoing advancements in AI technology, this approach promises even more robust watermarking techniques. This progress will enhance our ability to identify AI-generated content and provide an effective means for detecting plagiarism. + +[^footnote-3]: $\mathrm{e^{-i\pi}}$ diff --git a/papers/atharva_rasane/mybib.bib b/papers/atharva_rasane/mybib.bib new file mode 100644 index 0000000000..2296064be0 --- /dev/null +++ b/papers/atharva_rasane/mybib.bib @@ -0,0 +1,238 @@ +@book{book01, + author = "William Shakespeare", + year = "1998", + title = "Romeo and Juliet", + address = "USA", + url = {https://www.gutenberg.org/ebooks/1513}, +} + +@book{book02, + author = "Herman Melville", + year = "2001", + title = "Moby Dick; Or, The Whale", + address = "USA", + url = {https://www.gutenberg.org/ebooks/2701}, +} + +@book{book03, + author = "Jane Austen", + year = "1998", + title = "Pride and Prejudice", + address = "USA", + url = {https://www.gutenberg.org/ebooks/1342}, +} + +@book{book04, + author = "Mary Wollstonecraft Shelley", + year = "1993", + title = "Frankenstein; Or, The Modern Prometheus", + address = "USA", + url = {https://www.gutenberg.org/ebooks/84}, +} + +@book{book05, + author = "George Eliot", + year = "1994", + title = "Middlemarch", + address = "USA", + url = {https://www.gutenberg.org/ebooks/145}, +} + +@book{book06, + author = "William Shakespeare", + year = "1994", + title = "The Complete Works of William Shakespeare", + address = "USA", + url = {https://www.gutenberg.org/ebooks/100}, +} + +@book{book07, + author = "E. M. Forster", + year = "2001", + title = "A Room with a View", + address = "USA", + url = {https://www.gutenberg.org/ebooks/2641}, +} + +@book{book08, + author = "Louisa May Alcott", + year = "2011", + title = "Little Women; Or, Meg, Jo, Beth, and Amy", + address = "USA", + url = {https://www.gutenberg.org/ebooks/37106}, +} + +@book{book09, + author = "L. M. Montgomery", + year = "2022", + title = "The Blue Castle", + address = "USA", + url = {https://www.gutenberg.org/ebooks/67979}, +} + +@book{book10, + author = "Elizabeth Von Arnim", + year = "2005", + title = "The Enchanted April", + address = "USA", + url = {https://www.gutenberg.org/ebooks/16389}, +} + +@article{Atr01, + author = "Kamaruddin, Nurul Shamimi and Kamsin, Amirrudin and Por, Lip Yee and Rahman, Hameedur", + year = "2018", + title = "A Review of Text Watermarking: Theory, Methods, and Applications", + journal = "IEEE Access", + volume = 6, + issue = 3, + pages = {}, + doi = {10.1109/ACCESS.2018.2796585}} +} + +@article{Atr02, + author = "Lancaster, T", + year = "2023", + title = "Artificial intelligence, text generation tools and ChatGPT - does digital watermarking offer a solution?", + journal = "Int J Educ Integr", + volume = 19, + issue = 10, + pages = {8011-8028}, + doi = {https://doi.org/10.1007/s40979-023-00131-6} +} + +@article{Atr03, + author = "Yichao Wu and Zhengyu Jin and Chenxi Shi and Penghao Liang and Tong Zhan", + year = "2024", + title = "Research on the Application of Deep Learning-based BERT Model in Sentiment Analysis", + journal = "ArXiv", + volume = {abs/2403.08217}, + url = {https://api.semanticscholar.org/CorpusID:268379403} +} + +@article{Atr04, +title = {UniSpaCh: A text-based data hiding method using Unicode space characters}, +journal = {Journal of Systems and Software}, +volume = {85}, +number = {5}, +pages = {1075-1082}, +year = {2012}, +issn = {0164-1212}, +doi = {https://doi.org/10.1016/j.jss.2011.12.023}, +url = {https://www.sciencedirect.com/science/article/pii/S0164121211003177}, +author = {Lip Yee Por and KokSheik Wong and Kok Onn Chee}, +keywords = {UniSpaCh, DASH, Data hiding, Unicode character, Space manipulation}, +abstract = {This paper proposes a text-based data hiding method to insert external information into Microsoft Word document. First, the drawback of low embedding efficiency in the existing text-based data hiding methods is addressed, and a simple attack, DASH, is proposed to reveal the information inserted by the existing text-based data hiding methods. Then, a new data hiding method, UniSpaCh, is proposed to counter DASH. The characteristics of Unicode space characters with respect to embedding efficiency and DASH are analyzed, and the selected Unicode space characters are inserted into inter-sentence, inter-word, end-of-line and inter-paragraph spacings to encode external information while improving embedding efficiency and imperceptivity of the embedded information. UniSpaCh is also reversible where the embedded information can be removed to completely reconstruct the original Microsoft Word document. Experiments were carried out to verify the performance of UniSpaCh as well as comparing it to the existing space-manipulating data hiding methods. Results suggest that UniSpaCh offers higher embedding efficiency while exhibiting higher imperceptivity of white space manipulation when compared to the existing methods considered. In the best case scenario, UniSpaCh produces output document of size almost 9 times smaller than that of the existing method.} +} + +@INPROCEEDINGS{Proc01, + author={Jalil, Zunera and Mirza, Anwar M.}, + booktitle={2009 International Conference on Information and Multimedia Technology}, + title={A Review of Digital Watermarking Techniques for Text Documents}, + year={2009}, + volume={}, + number={}, + pages={230-234}, + keywords={Watermarking;Copyright protection;Internet;Cryptography;Steganography;Computer science;Information security;Intellectual property;Data mining;Law;watermarking;copyright protection;information security;text structure}, + doi={10.1109/ICIMT.2009.11}} + +# These references may be helpful: + +@inproceedings{jupyter, + abstract = {It is increasingly necessary for researchers in all fields to write computer code, and in order to reproduce research results, it is important that this code is published. We present Jupyter notebooks, a document format for publishing code, results and explanations in a form that is both readable and executable. We discuss various tools and use cases for notebook documents.}, + author = {Kluyver, Thomas and Ragan-Kelley, Benjamin and Pérez, Fernando and Granger, Brian and Bussonnier, Matthias and Frederic, Jonathan and Kelley, Kyle and Hamrick, Jessica and Grout, Jason and Corlay, Sylvain and Ivanov, Paul and Avila, Damián and Abdalla, Safia and Willing, Carol and {Jupyter development team}}, + editor = {Loizides, Fernando and Scmidt, Birgit}, + location = {Netherlands}, + publisher = {IOS Press}, + url = {https://eprints.soton.ac.uk/403913/}, + booktitle = {Positioning and Power in Academic Publishing: Players, Agents and Agendas}, + year = {2016}, + pages = {87--90}, + title = {Jupyter Notebooks - a publishing format for reproducible computational workflows}, +} + +@article{matplotlib, + abstract = {Matplotlib is a 2D graphics package used for Python for application development, interactive scripting, and publication-quality image generation across user interfaces and operating systems.}, + author = {Hunter, J. D.}, + publisher = {IEEE COMPUTER SOC}, + year = {2007}, + doi = {https://doi.org/10.1109/MCSE.2007.55}, + journal = {Computing in Science \& Engineering}, + number = {3}, + pages = {90--95}, + title = {Matplotlib: A 2D graphics environment}, + volume = {9}, +} + +@article{numpy, + author = {Harris, Charles R. and Millman, K. Jarrod and van der Walt, Stéfan J. and Gommers, Ralf and Virtanen, Pauli and Cournapeau, David and Wieser, Eric and Taylor, Julian and Berg, Sebastian and Smith, Nathaniel J. and Kern, Robert and Picus, Matti and Hoyer, Stephan and van Kerkwijk, Marten H. and Brett, Matthew and Haldane, Allan and del Río, Jaime Fernández and Wiebe, Mark and Peterson, Pearu and Gérard-Marchant, Pierre and Sheppard, Kevin and Reddy, Tyler and Weckesser, Warren and Abbasi, Hameer and Gohlke, Christoph and Oliphant, Travis E.}, + publisher = {Springer Science and Business Media {LLC}}, + doi = {https://doi.org/10.1038/s41586-020-2649-2}, + date = {2020-09}, + year = {2020}, + journal = {Nature}, + number = {7825}, + pages = {357--362}, + title = {Array programming with {NumPy}}, + volume = {585}, +} + +@misc{pandas1, + author = {{The Pandas Development Team}}, + title = {pandas-dev/pandas: Pandas}, + month = feb, + year = {2020}, + publisher = {Zenodo}, + version = {latest}, + url = {https://doi.org/10.5281/zenodo.3509134}, +} + +@inproceedings{pandas2, + author = {Wes McKinney}, + title = {{D}ata {S}tructures for {S}tatistical {C}omputing in {P}ython}, + booktitle = {{P}roceedings of the 9th {P}ython in {S}cience {C}onference}, + pages = {56 - 61}, + year = {2010}, + editor = {{S}t\'efan van der {W}alt and {J}arrod {M}illman}, + doi = {https://doi.org/10.25080/Majora-92bf1922-00a}, +} + +@article{scipy, + author = {Virtanen, Pauli and Gommers, Ralf and Oliphant, Travis E. and + Haberland, Matt and Reddy, Tyler and Cournapeau, David and + Burovski, Evgeni and Peterson, Pearu and Weckesser, Warren and + Bright, Jonathan and {van der Walt}, St{\'e}fan J. and + Brett, Matthew and Wilson, Joshua and Millman, K. Jarrod and + Mayorov, Nikolay and Nelson, Andrew R. J. and Jones, Eric and + Kern, Robert and Larson, Eric and Carey, C J and + Polat, {\.I}lhan and Feng, Yu and Moore, Eric W. and + {VanderPlas}, Jake and Laxalde, Denis and Perktold, Josef and + Cimrman, Robert and Henriksen, Ian and Quintero, E. A. and + Harris, Charles R. and Archibald, Anne M. and + Ribeiro, Ant{\^o}nio H. and Pedregosa, Fabian and + {van Mulbregt}, Paul and {SciPy 1.0 Contributors}}, + title = {{{SciPy} 1.0: Fundamental Algorithms for Scientific + Computing in Python}}, + journal = {Nature Methods}, + year = {2020}, + volume = {17}, + pages = {261--272}, + adsurl = {https://rdcu.be/b08Wh}, + doi = {https://doi.org/10.1038/s41592-019-0686-2}, +} + +@article{sklearn1, + author = {Pedregosa, F. and Varoquaux, G. and Gramfort, A. and Michel, V. and Thirion, B. and Grisel, O. and Blondel, M. and Prettenhofer, P. and Weiss, R. and Dubourg, V. and Vanderplas, J. and Passos, A. and Cournapeau, D. and Brucher, M. and Perrot, M. and Duchesnay, E.}, + year = {2011}, + journal = {Journal of Machine Learning Research}, + pages = {2825--2830}, + title = {Scikit-learn: Machine Learning in {P}ython}, + volume = {12}, +} + +@inproceedings{sklearn2, + author = {Buitinck, Lars and Louppe, Gilles and Blondel, Mathieu and Pedregosa, Fabian and Mueller, Andreas and Grisel, Olivier and Niculae, Vlad and Prettenhofer, Peter and Gramfort, Alexandre and Grobler, Jaques and Layton, Robert and VanderPlas, Jake and Joly, Arnaud and Holt, Brian and Varoquaux, Gaël}, + booktitle = {ECML PKDD Workshop: Languages for Data Mining and Machine Learning}, + year = {2013}, + pages = {108--122}, + title = {{API} design for machine learning software: experiences from the scikit-learn project}, +} diff --git a/papers/atharva_rasane/myst.yml b/papers/atharva_rasane/myst.yml new file mode 100644 index 0000000000..5527e8f92f --- /dev/null +++ b/papers/atharva_rasane/myst.yml @@ -0,0 +1,57 @@ +version: 1 +project: + # Update this to match `scipy-2024-` the folder should be `` + id: scipy-2024-atharva_rasane + # Ensure your title is the same as in your `main.md` + title: AI-Driven Watermarking Technique for Safeguarding Text Integrity in the Digital Age + subtitle: + # Authors should have affiliations, emails and ORCIDs if available + authors: + - name: Atharva Rasane + email: rratharva@gmail.com + affiliations: + - KLE Technology University + keywords: + - physical watermark + - logical watermark + - HuggingFace Transformer Library + - BERT + # Add the abbreviations that you use in your paper here + abbreviations: + BERT: Bidirectional Encoder Representations from Transformers + AI: Artificial Intelligence + NLP: Natural Language Processing + NSP: Next Sentence Prediction + GPU: Graphics Processing Units + T-statistics: Test Statistics + P-value: Probability Value + DRM: Digital Rights Management + CMS: Content Management Systems + # It is possible to explicitly ignore the `doi-exists` check for certain citation keys + error_rules: + - rule: doi-exists + severity: ignore + keys: + - book01 + - book02 + - book03 + - book04 + - book05 + - book06 + - book07 + - book08 + - book09 + - book10 + - Atr01 + - Atr02 + - Atr03 + # A banner will be generated for you on publication, this is a placeholder + banner: banner.png + # The rest of the information shouldn't be modified + subject: Research Article + open_access: true + license: CC-BY-4.0 + venue: Scipy 2024 + date: 2024-07-10 +site: + template: article-theme